2024-05-27T06:28:40.646 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T06:28:40.646 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_vshankar@teuthology --name vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} --archive /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619 -- /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/orig.config.yaml 2024-05-27T06:28:40.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T06:28:40.811 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619 branch: wip-vshankar-testing-20240521.063558-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} email: vshankar@redhat.com first_in_suite: false job_id: '7727619' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.063558-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.063558-debug sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2499 sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 sleep_before_teardown: 0 subset: 573/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.063558-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA8icj0uRzTjbm3Xx+WPvwSmRsuqQhf+jqflAGS7fYWaWqlRjokKWgbnZaSOUDZgy5g3Dfjr30x0CDxzx1+dSZM= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCH86YcLMjbtpDCSFu4xoz4O3SmzaPcEUYCN/+oGkMaJ0z+N3f6FjXlgeCaNDUAJJIEK1TAApCummDUUIr67P4c= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWGWQ4gwlhBJ8R3a/qMWs0XcqGgUnV3QD9pzgurqsQCR7UlsyY4+n3ptVoL1WeNdi+1/i/aKrrWU6jimq8TyEY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_05:52:49 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T06:28:40.812 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa; will attempt to use it 2024-05-27T06:28:40.812 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks 2024-05-27T06:28:40.813 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T06:28:40.814 INFO:teuthology.task.internal:Checking packages... 2024-05-27T06:28:40.835 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '38f9d900c83c63799fbdbe61acc9a11b0d3554a6' 2024-05-27T06:28:40.836 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T06:28:40.836 INFO:teuthology.packaging:ref: None 2024-05-27T06:28:40.836 INFO:teuthology.packaging:tag: None 2024-05-27T06:28:40.836 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.063558-debug 2024-05-27T06:28:40.836 INFO:teuthology.packaging:sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:28:40.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.063558-debug 2024-05-27T06:28:40.977 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3872.g38f9d900 2024-05-27T06:28:40.979 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T06:28:40.985 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T06:28:40.985 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T06:28:41.001 INFO:teuthology.task.internal:Saving configuration 2024-05-27T06:28:41.022 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T06:28:41.029 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T06:28:41.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:23:34.457458', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA8icj0uRzTjbm3Xx+WPvwSmRsuqQhf+jqflAGS7fYWaWqlRjokKWgbnZaSOUDZgy5g3Dfjr30x0CDxzx1+dSZM='} 2024-05-27T06:28:41.070 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:23:34.453830', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCH86YcLMjbtpDCSFu4xoz4O3SmzaPcEUYCN/+oGkMaJ0z+N3f6FjXlgeCaNDUAJJIEK1TAApCummDUUIr67P4c='} 2024-05-27T06:28:41.091 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:23:34.455734', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWGWQ4gwlhBJ8R3a/qMWs0XcqGgUnV3QD9pzgurqsQCR7UlsyY4+n3ptVoL1WeNdi+1/i/aKrrWU6jimq8TyEY='} 2024-05-27T06:28:41.092 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T06:28:41.098 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-27T06:28:41.098 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-27T06:28:41.098 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-27T06:28:41.098 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T06:28:41.178 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6e72fb3ac0>, signals=[15]) 2024-05-27T06:28:41.178 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T06:28:41.184 INFO:teuthology.task.internal:Opening connections... 2024-05-27T06:28:41.185 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T06:28:41.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:28:41.266 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-05-27T06:28:41.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:28:41.341 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-05-27T06:28:41.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:28:41.416 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T06:28:41.422 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-05-27T06:28:41.441 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-05-27T06:28:41.441 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-05-27T06:28:41.499 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-05-27T06:28:41.499 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-05-27T06:28:41.499 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T06:28:41.500 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T06:28:41.501 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-05-27T06:28:41.520 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-05-27T06:28:41.537 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-05-27T06:28:41.537 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-05-27T06:28:41.594 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T06:28:41.595 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T06:28:41.595 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-05-27T06:28:41.620 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-05-27T06:28:41.638 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-05-27T06:28:41.638 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-05-27T06:28:41.695 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-05-27T06:28:41.695 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-05-27T06:28:41.695 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-05-27T06:28:41.695 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T06:28:41.696 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T06:28:41.696 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-05-27T06:28:41.720 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T06:28:41.730 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T06:28:41.740 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T06:28:41.741 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-05-27T06:28:41.743 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-05-27T06:28:41.745 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-05-27T06:28:41.761 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T06:28:41.767 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T06:28:41.768 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-05-27T06:28:41.801 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-05-27T06:28:41.803 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-05-27T06:28:41.922 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T06:28:41.994 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T06:28:42.008 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T06:28:42.008 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-27T06:28:42.008 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T06:28:42.009 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T06:28:42.010 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T06:28:42.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:28:42.027 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.027 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T06:28:42.027 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-27T06:28:42.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:28:42.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:28:42.029 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.029 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T06:28:42.029 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-05-27T06:28:42.029 DEBUG:teuthology.task.kernel:sha1 for mon.c is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.029 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T06:28:42.029 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-05-27T06:28:42.083 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-05-27T06:28:42.084 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.084 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T06:28:42.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.089 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-437.el9.x86_64 2024-05-27T06:28:42.089 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-437.el9.x86_64 2024-05-27T06:28:42.089 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.089 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T06:28:42.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.090 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.091 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T06:28:42.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:28:42.227 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum upgrade -y linux-firmware 2024-05-27T06:28:42.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum upgrade -y linux-firmware 2024-05-27T06:28:42.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2024-05-27T06:28:44.276 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-27T06:28:44.326 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-27T06:28:45.985 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-27T06:28:46.760 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-05-27T06:28:46.874 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:02 2024-05-27T06:28:50.294 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-05-27T06:28:51.811 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 17 kB 00:00 2024-05-27T06:28:52.092 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-05-27T06:28:52.127 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-27T06:28:52.971 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-27T06:28:53.661 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 17 kB 00:00 2024-05-27T06:28:55.698 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-27T06:28:57.217 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-27T06:28:58.112 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-05-27T06:28:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 17 kB 00:00 2024-05-27T06:28:59.536 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 32 MB/s | 21 MB 00:00 2024-05-27T06:29:00.120 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-27T06:29:00.121 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-05-27T06:29:00.121 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-27T06:29:00.217 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T06:29:00.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:29:00.375 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T06:29:00.375 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T06:29:00.378 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi069.front.sepia.ceph.com to complete... 2024-05-27T06:29:00.845 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-27T06:29:02.831 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-05-27T06:29:02.832 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-05-27T06:29:02.832 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-05-27T06:29:02.936 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T06:29:02.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:29:03.163 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T06:29:03.163 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T06:29:03.165 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi164.front.sepia.ceph.com to complete... 2024-05-27T06:29:04.811 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-27T06:29:06.906 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-27T06:29:06.907 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-05-27T06:29:06.907 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-27T06:29:06.991 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T06:29:06.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T06:29:07.215 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T06:29:07.215 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T06:29:07.217 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-05-27T06:29:18.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27 06:29:18 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T06:29:18.673 INFO:teuthology.orchestra.run.smithi069.stderr:FINISHED --2024-05-27 06:29:18-- 2024-05-27T06:29:18.673 INFO:teuthology.orchestra.run.smithi069.stderr:Total wall clock time: 18s 2024-05-27T06:29:18.674 INFO:teuthology.orchestra.run.smithi069.stderr:Downloaded: 1 files, 340M in 18s (19.0 MB/s) 2024-05-27T06:29:18.676 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T06:29:18.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T06:29:20.769 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying... ######################################## 2024-05-27T06:29:21.003 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing... ######################################## 2024-05-27T06:29:21.111 INFO:teuthology.orchestra.run.smithi069.stdout:Updating / installing... 2024-05-27T06:29:29.042 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T06:29:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27 06:29:36 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T06:29:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-05-27 06:29:36-- 2024-05-27T06:29:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 29s 2024-05-27T06:29:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 340M in 29s (11.9 MB/s) 2024-05-27T06:29:36.298 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T06:29:36.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T06:29:36.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27 06:29:36 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T06:29:36.464 INFO:teuthology.orchestra.run.smithi164.stderr:FINISHED --2024-05-27 06:29:36-- 2024-05-27T06:29:36.464 INFO:teuthology.orchestra.run.smithi164.stderr:Total wall clock time: 33s 2024-05-27T06:29:36.464 INFO:teuthology.orchestra.run.smithi164.stderr:Downloaded: 1 files, 340M in 33s (10.4 MB/s) 2024-05-27T06:29:36.468 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T06:29:36.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T06:29:38.485 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2024-05-27T06:29:38.555 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying... ######################################## 2024-05-27T06:29:38.740 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing... ######################################## 2024-05-27T06:29:38.840 INFO:teuthology.orchestra.run.smithi164.stdout:Updating / installing... 2024-05-27T06:29:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2024-05-27T06:29:40.543 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2024-05-27T06:29:46.817 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T06:29:48.865 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T06:30:07.707 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T06:30:07.708 DEBUG:teuthology.orchestra.run.smithi069:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:07.780 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T06:30:07.781 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T06:30:07.782 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T06:30:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T06:30:07.784 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T06:30:07.785 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T06:30:07.786 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T06:30:07.787 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T06:30:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T06:30:07.789 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T06:30:07.790 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T06:30:07.791 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T06:30:07.792 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T06:30:07.793 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T06:30:07.794 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T06:30:07.795 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T06:30:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T06:30:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T06:30:07.798 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T06:30:07.799 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T06:30:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T06:30:07.801 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T06:30:07.802 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T06:30:07.803 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T06:30:07.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T06:30:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T06:30:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T06:30:07.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T06:30:07.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T06:30:07.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T06:30:07.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T06:30:07.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T06:30:07.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T06:30:07.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T06:30:07.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T06:30:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T06:30:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T06:30:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T06:30:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T06:30:07.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T06:30:07.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T06:30:07.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T06:30:07.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T06:30:07.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T06:30:07.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T06:30:07.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T06:30:07.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T06:30:07.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T06:30:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T06:30:07.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T06:30:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T06:30:07.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T06:30:07.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T06:30:07.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T06:30:07.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T06:30:07.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T06:30:07.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T06:30:07.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T06:30:07.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T06:30:07.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T06:30:07.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T06:30:07.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T06:30:07.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T06:30:07.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T06:30:07.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T06:30:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T06:30:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T06:30:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T06:30:07.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T06:30:07.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T06:30:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T06:30:07.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T06:30:07.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T06:30:07.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T06:30:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T06:30:07.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T06:30:07.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T06:30:07.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T06:30:07.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T06:30:07.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T06:30:07.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T06:30:07.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T06:30:07.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T06:30:07.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T06:30:07.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T06:30:07.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T06:30:07.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T06:30:07.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T06:30:07.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T06:30:07.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T06:30:07.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T06:30:07.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T06:30:07.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T06:30:07.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T06:30:07.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T06:30:07.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T06:30:07.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T06:30:07.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T06:30:07.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T06:30:07.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T06:30:07.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T06:30:07.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T06:30:07.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T06:30:07.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T06:30:07.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T06:30:07.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T06:30:07.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T06:30:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T06:30:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T06:30:07.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T06:30:07.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T06:30:07.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T06:30:07.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T06:30:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T06:30:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T06:30:07.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T06:30:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T06:30:07.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T06:30:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T06:30:07.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T06:30:07.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T06:30:07.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T06:30:07.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T06:30:07.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T06:30:07.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T06:30:07.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T06:30:07.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T06:30:07.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T06:30:07.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T06:30:07.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T06:30:07.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T06:30:07.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T06:30:07.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T06:30:07.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T06:30:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T06:30:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T06:30:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T06:30:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T06:30:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T06:30:07.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T06:30:07.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T06:30:07.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T06:30:07.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T06:30:07.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T06:30:07.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T06:30:07.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:07.927 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T06:30:07.928 DEBUG:teuthology.orchestra.run.smithi069:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:for file in vmlinuz System.map config; do 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: fi 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:done 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T06:30:07.950 INFO:teuthology.orchestra.run.smithi069.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T06:30:07.951 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-05-27T06:30:07.952 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-05-27T06:30:08.032 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T06:30:08.033 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-05-27T06:30:08.034 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:30:08.034 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T06:30:08.034 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T06:30:08.035 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 6.9.0-gb9459a26b088 2024-05-27T06:30:08.035 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T06:30:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-05-27T06:30:10.016 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T06:30:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-05-27T06:30:10.058 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-05-27T06:30:10.074 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.2W0Wv3GSbH 2024-05-27T06:30:10.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2W0Wv3GSbH 2024-05-27T06:30:10.140 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.2W0Wv3GSbH 2024-05-27T06:30:10.276 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.2W0Wv3GSbH is 6KB 2024-05-27T06:30:10.326 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.2W0Wv3GSbH 2024-05-27T06:30:10.340 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T06:30:10.410 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T06:30:10.410 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T06:30:10.410 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T06:30:10.410 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T06:30:10.411 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T06:30:10.558 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-05-27T06:30:24.120 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T06:30:24.120 DEBUG:teuthology.orchestra.run.smithi164:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:24.197 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T06:30:24.197 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T06:30:24.197 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T06:30:24.197 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T06:30:24.198 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T06:30:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T06:30:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T06:30:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T06:30:24.202 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T06:30:24.203 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T06:30:24.204 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T06:30:24.205 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T06:30:24.206 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T06:30:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T06:30:24.208 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T06:30:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T06:30:24.210 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T06:30:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T06:30:24.212 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T06:30:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T06:30:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T06:30:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T06:30:24.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T06:30:24.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T06:30:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T06:30:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T06:30:24.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T06:30:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T06:30:24.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T06:30:24.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T06:30:24.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T06:30:24.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T06:30:24.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T06:30:24.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T06:30:24.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T06:30:24.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T06:30:24.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T06:30:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T06:30:24.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T06:30:24.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T06:30:24.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T06:30:24.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T06:30:24.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T06:30:24.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T06:30:24.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T06:30:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T06:30:24.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T06:30:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T06:30:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T06:30:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T06:30:24.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T06:30:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T06:30:24.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T06:30:24.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T06:30:24.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T06:30:24.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T06:30:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T06:30:24.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T06:30:24.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T06:30:24.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T06:30:24.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T06:30:24.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T06:30:24.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T06:30:24.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T06:30:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T06:30:24.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T06:30:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T06:30:24.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T06:30:24.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T06:30:24.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T06:30:24.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T06:30:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T06:30:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T06:30:24.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T06:30:24.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T06:30:24.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T06:30:24.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T06:30:24.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T06:30:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T06:30:24.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T06:30:24.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T06:30:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T06:30:24.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T06:30:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T06:30:24.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T06:30:24.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T06:30:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T06:30:24.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T06:30:24.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T06:30:24.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T06:30:24.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T06:30:24.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T06:30:24.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T06:30:24.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T06:30:24.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T06:30:24.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T06:30:24.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T06:30:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T06:30:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T06:30:24.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T06:30:24.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T06:30:24.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T06:30:24.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T06:30:24.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T06:30:24.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T06:30:24.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T06:30:24.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T06:30:24.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T06:30:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T06:30:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T06:30:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T06:30:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T06:30:24.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T06:30:24.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T06:30:24.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T06:30:24.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T06:30:24.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T06:30:24.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T06:30:24.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T06:30:24.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T06:30:24.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T06:30:24.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T06:30:24.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T06:30:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T06:30:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T06:30:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T06:30:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T06:30:24.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T06:30:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T06:30:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T06:30:24.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T06:30:24.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T06:30:24.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T06:30:24.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T06:30:24.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T06:30:24.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T06:30:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T06:30:24.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T06:30:24.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T06:30:24.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T06:30:24.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T06:30:24.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T06:30:24.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T06:30:24.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T06:30:24.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T06:30:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:24.343 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T06:30:24.343 DEBUG:teuthology.orchestra.run.smithi164:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout:for file in vmlinuz System.map config; do 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T06:30:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: fi 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T06:30:24.366 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-05-27T06:30:24.367 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 68.el9 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8264254 2024-05-27T06:30:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-05-27T06:30:24.440 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:30:24.441 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T06:30:24.441 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T06:30:24.441 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 6.9.0-gb9459a26b088 2024-05-27T06:30:24.442 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T06:30:25.058 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-05-27T06:30:26.429 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T06:30:26.457 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-05-27T06:30:26.458 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-05-27T06:30:26.476 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.QVnQVbavQc 2024-05-27T06:30:26.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QVnQVbavQc 2024-05-27T06:30:26.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.QVnQVbavQc 2024-05-27T06:30:26.692 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.QVnQVbavQc is 6KB 2024-05-27T06:30:26.740 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.QVnQVbavQc 2024-05-27T06:30:26.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T06:30:26.815 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T06:30:26.815 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T06:30:26.815 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T06:30:26.815 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T06:30:26.816 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T06:30:26.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-05-27T06:30:27.177 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T06:30:27.177 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:27.268 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T06:30:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T06:30:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T06:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T06:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T06:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T06:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T06:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T06:30:27.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T06:30:27.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T06:30:27.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T06:30:27.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T06:30:27.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T06:30:27.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T06:30:27.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T06:30:27.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T06:30:27.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T06:30:27.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T06:30:27.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T06:30:27.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T06:30:27.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T06:30:27.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T06:30:27.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T06:30:27.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T06:30:27.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T06:30:27.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T06:30:27.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T06:30:27.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T06:30:27.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T06:30:27.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T06:30:27.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T06:30:27.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T06:30:27.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T06:30:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T06:30:27.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T06:30:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T06:30:27.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T06:30:27.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T06:30:27.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T06:30:27.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T06:30:27.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T06:30:27.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T06:30:27.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T06:30:27.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T06:30:27.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T06:30:27.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T06:30:27.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T06:30:27.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T06:30:27.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T06:30:27.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T06:30:27.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T06:30:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T06:30:27.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T06:30:27.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T06:30:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T06:30:27.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T06:30:27.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T06:30:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T06:30:27.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T06:30:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T06:30:27.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T06:30:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T06:30:27.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T06:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T06:30:27.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T06:30:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T06:30:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T06:30:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T06:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T06:30:27.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T06:30:27.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T06:30:27.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T06:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T06:30:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T06:30:27.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T06:30:27.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T06:30:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T06:30:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T06:30:27.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T06:30:27.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T06:30:27.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T06:30:27.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T06:30:27.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T06:30:27.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T06:30:27.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T06:30:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T06:30:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T06:30:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T06:30:27.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T06:30:27.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T06:30:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T06:30:27.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T06:30:27.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T06:30:27.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T06:30:27.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T06:30:27.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T06:30:27.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T06:30:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T06:30:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T06:30:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T06:30:27.348 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T06:30:27.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T06:30:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T06:30:27.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T06:30:27.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T06:30:27.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T06:30:27.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T06:30:27.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T06:30:27.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T06:30:27.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T06:30:27.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T06:30:27.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T06:30:27.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T06:30:27.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T06:30:27.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T06:30:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T06:30:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T06:30:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T06:30:27.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T06:30:27.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T06:30:27.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T06:30:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T06:30:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T06:30:27.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T06:30:27.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T06:30:27.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T06:30:27.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T06:30:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T06:30:27.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T06:30:27.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T06:30:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T06:30:27.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T06:30:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T06:30:27.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T06:30:27.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T06:30:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T06:30:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T06:30:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T06:30:27.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T06:30:27.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T06:30:27.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T06:30:27.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T06:30:27.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T06:30:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T06:30:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T06:30:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T06:30:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T06:30:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T06:30:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T06:30:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T06:30:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T06:30:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T06:30:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T06:30:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T06:30:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T06:30:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T06:30:27.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T06:30:27.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T06:30:27.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T06:30:27.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T06:30:27.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T06:30:27.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T06:30:27.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T06:30:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T06:30:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T06:30:27.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T06:30:27.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T06:30:27.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T06:30:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T06:30:27.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T06:30:27.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T06:30:27.419 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T06:30:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T06:30:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T06:30:27.422 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T06:30:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T06:30:27.424 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T06:30:27.425 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T06:30:27.426 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T06:30:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T06:30:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T06:30:27.429 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T06:30:27.431 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T06:30:27.434 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:27.440 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T06:30:27.440 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T06:30:27.466 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T06:30:27.466 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:27.466 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T06:30:27.466 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-05-27T06:30:27.466 INFO:teuthology.orchestra.run.smithi016.stdout:for file in vmlinuz System.map config; do 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: fi 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T06:30:27.467 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T06:30:27.468 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-05-27T06:30:27.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T06:30:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:30:27.547 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T06:30:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T06:30:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T06:30:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-05-27T06:30:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:30:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T06:30:27.549 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T06:30:27.549 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.9.0-gb9459a26b088 2024-05-27T06:30:27.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T06:30:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-05-27T06:30:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T06:30:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-05-27T06:30:29.717 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-05-27T06:30:29.748 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.QluIj9SjGG 2024-05-27T06:30:29.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QluIj9SjGG 2024-05-27T06:30:29.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.QluIj9SjGG 2024-05-27T06:30:29.960 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.QluIj9SjGG is 6KB 2024-05-27T06:30:30.009 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.QluIj9SjGG 2024-05-27T06:30:30.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T06:30:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T06:30:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T06:30:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T06:30:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T06:30:30.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T06:30:30.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-05-27T06:30:40.563 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-27T06:30:40.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:30:40.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:30:56.978 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-27T06:30:56.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:30:56.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:30:59.106 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-05-27T06:31:00.271 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-27T06:31:00.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:31:00.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:08.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:31:08.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:11.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-05-27T06:31:15.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-27T06:31:18.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-05-27T06:31:23.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:31:23.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:24.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:31:24.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:27.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-27T06:31:27.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:31:27.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:39.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:31:39.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:41.602 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-05-27T06:31:56.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:31:56.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:31:57.079 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-27T06:31:57.735 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:31:57.736 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-27T06:31:57.736 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-27T06:31:57.776 INFO:teuthology.orchestra.run.smithi069.stdout:6.9.0-gb9459a26b088 2024-05-27T06:31:57.777 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T06:31:57.777 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T06:31:57.777 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T06:31:57.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-27T06:31:58.777 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-27T06:31:58.778 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T06:31:58.895 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2024-05-27T06:31:58.933 DEBUG:teuthology.parallel:result is None 2024-05-27T06:32:12.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:32:12.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:32:16.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-27T06:32:27.823 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T06:32:34.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:32:34.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:32:34.505 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-05-27T06:32:35.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:32:35.245 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-gb9459a26b088"... 2024-05-27T06:32:35.245 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-05-27T06:32:35.282 INFO:teuthology.orchestra.run.smithi164.stdout:6.9.0-gb9459a26b088 2024-05-27T06:32:35.282 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T06:32:35.282 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T06:32:35.283 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T06:32:36.284 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-27T06:32:36.284 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T06:32:36.396 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-05-27T06:32:36.451 DEBUG:teuthology.parallel:result is None 2024-05-27T06:32:39.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:32:39.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:32:40.284 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-05-27T06:32:40.924 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:32:40.924 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-27T06:32:40.924 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-05-27T06:32:40.981 INFO:teuthology.orchestra.run.smithi016.stdout:6.9.0-gb9459a26b088 2024-05-27T06:32:40.981 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T06:32:40.981 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T06:32:40.981 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T06:32:41.982 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-27T06:32:41.983 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T06:32:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-05-27T06:32:42.189 DEBUG:teuthology.parallel:result is None 2024-05-27T06:32:42.190 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T06:32:42.199 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T06:32:42.199 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T06:32:42.206 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T06:32:42.215 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T06:32:42.273 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T06:32:42.285 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T06:32:42.298 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T06:32:42.298 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T06:32:42.304 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T06:32:42.309 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T06:32:42.391 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T06:32:42.402 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T06:32:42.402 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T06:32:42.410 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T06:32:42.416 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T06:32:42.508 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.510 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.512 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.538 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.542 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.546 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T06:32:42.551 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T06:32:42.562 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T06:32:42.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T06:32:42.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T06:32:42.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T06:32:42.658 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T06:32:42.669 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T06:32:42.669 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T06:32:42.674 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T06:32:42.680 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T06:32:42.734 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:42.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:42.922 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.097 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:32:43.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T06:32:43.185 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:43.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:43.369 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.454 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.540 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:32:43.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T06:32:43.620 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:43.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T06:32:43.804 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.886 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T06:32:43.970 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:32:43.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T06:32:44.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-05-27T06:32:44.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-05-27T06:32:44.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-05-27T06:32:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T06:32:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T06:32:44.219 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T06:32:44.717 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T06:32:44.729 INFO:teuthology.task.internal:Starting timer... 2024-05-27T06:32:44.729 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T06:32:44.751 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T06:32:44.762 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T06:32:44.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-05-27T06:32:44.891 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-05-27T06:32:44.896 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-05-27T06:32:45.037 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-05-27T06:32:45.042 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-05-27T06:32:45.166 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-05-27T06:32:45.169 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T06:32:45.170 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-05-27T06:32:45.278 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-05-27T06:32:45.280 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-05-27T06:32:45.350 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-05-27T06:32:45.350 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-05-27T06:32:45.411 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-05-27T06:32:45.411 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-05-27T06:32:45.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T06:32:45.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:32:45.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T06:32:45.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:32:45.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T06:32:45.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T06:32:45.645 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T06:32:45.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-05-27T06:32:45.724 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-05-27T06:32:45.790 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-05-27T06:32:45.868 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T06:32:45.878 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-27T06:32:45.882 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-27T06:32:46.048 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T06:32:46.087 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T06:32:46.088 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi069.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T06:39:26.578 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-05-27T06:39:26.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:39:26.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:39:26.694 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-05-27T06:39:26.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-27T06:39:26.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:39:26.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:39:26.961 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-27T06:39:27.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-27T06:39:27.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:39:27.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T06:39:27.237 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-05-27T06:39:27.388 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-27T06:39:27.389 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T06:39:27.400 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T06:39:27.400 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T06:39:27.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T06:39:27.407 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T06:39:27.407 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T06:39:27.414 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T06:39:27.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T06:39:27.501 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T06:39:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T06:39:27.508 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T06:39:27.542 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T06:39:27.549 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T06:39:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T06:39:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-05-27T06:39:27.622 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-05-27T06:39:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-05-27T06:39:27.650 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-05-27T06:39:27.656 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-05-27T06:39:27.666 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-05-27T06:39:27.690 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T06:39:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T06:39:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T06:39:27.735 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T06:39:27.737 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T06:39:27.758 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T06:39:27.871 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-05-27T06:39:27.875 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.880 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.886 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T06:39:27.886 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-27T06:39:27.886 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.887 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.887 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.887 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-05-27T06:39:27.911 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T06:39:27.911 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-27T06:39:27.911 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.911 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.912 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.912 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T06:39:27.913 INFO:teuthology.run_tasks:Running task install... 2024-05-27T06:39:27.922 DEBUG:teuthology.task.install:project ceph 2024-05-27T06:39:27.922 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'}} 2024-05-27T06:39:27.922 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'} 2024-05-27T06:39:27.922 INFO:teuthology.task.install:Using flavor: default 2024-05-27T06:39:27.928 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T06:39:27.928 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T06:39:27.928 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T06:39:27.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:39:27.931 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T06:39:27.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:39:27.932 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T06:39:27.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:39:28.078 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T06:39:28.079 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T06:39:28.088 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T06:39:28.088 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T06:39:28.146 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T06:39:28.146 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T06:39:28.195 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T06:39:28.195 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:39:28.195 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T06:39:28.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T06:39:28.258 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:39:28.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T06:39:28.265 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T06:39:28.266 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:39:28.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T06:39:28.283 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T06:39:28.283 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T06:39:28.355 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T06:39:28.356 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T06:39:28.357 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T06:39:28.357 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T06:39:28.414 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T06:39:28.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T06:39:28.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T06:39:28.520 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T06:39:28.592 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T06:39:28.602 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-05-27T06:39:28.606 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-05-27T06:39:28.611 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T06:39:28.671 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-05-27T06:39:28.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-05-27T06:39:28.693 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-05-27T06:39:28.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-05-27T06:39:28.922 INFO:teuthology.orchestra.run.smithi069.stdout:45 files removed 2024-05-27T06:39:28.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T06:39:29.012 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-05-27T06:39:29.028 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-05-27T06:39:29.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T06:39:29.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T06:39:29.877 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 213 kB/s | 84 kB 00:00 2024-05-27T06:39:29.939 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 200 kB/s | 84 kB 00:00 2024-05-27T06:39:29.964 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 149 kB/s | 84 kB 00:00 2024-05-27T06:39:30.439 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-27T06:39:30.444 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-27T06:39:30.548 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-27T06:39:30.915 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-27T06:39:31.012 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-27T06:39:31.073 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-05-27T06:39:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-27T06:39:33.151 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-27T06:39:33.414 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.1 MB 00:02 2024-05-27T06:39:36.344 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 19 MB 00:01 2024-05-27T06:39:42.647 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 19 MB 00:09 2024-05-27T06:39:43.928 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-27T06:39:45.695 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 17 kB 00:00 2024-05-27T06:39:47.056 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-27T06:39:49.899 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 17 kB 00:01 2024-05-27T06:39:50.716 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-27T06:39:51.225 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-27T06:39:56.918 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-27T06:39:57.441 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-05-27T06:39:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 17 kB 00:00 2024-05-27T06:39:59.825 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:39:59.827 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:39:59.847 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T06:39:59.848 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T06:39:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T06:39:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T06:39:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T06:39:59.857 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T06:39:59.858 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T06:39:59.929 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T06:39:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T06:39:59.957 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T06:39:59.958 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T06:39:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T06:39:59.960 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T06:39:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T06:39:59.962 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T06:39:59.963 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T06:39:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T06:39:59.965 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T06:39:59.966 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T06:39:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T06:39:59.968 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 11 Packages 2024-05-27T06:39:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:39:59.970 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 219 M 2024-05-27T06:39:59.970 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-05-27T06:40:00.495 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-27T06:40:01.026 INFO:teuthology.orchestra.run.smithi069.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-05-27T06:40:01.218 INFO:teuthology.orchestra.run.smithi069.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 4.4 MB/s | 863 kB 00:00 2024-05-27T06:40:01.293 INFO:teuthology.orchestra.run.smithi069.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 1.9 MB/s | 147 kB 00:00 2024-05-27T06:40:01.618 INFO:teuthology.orchestra.run.smithi069.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 7.0 MB/s | 2.3 MB 00:00 2024-05-27T06:40:01.844 INFO:teuthology.orchestra.run.smithi069.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-05-27T06:40:02.220 INFO:teuthology.orchestra.run.smithi069.stdout:(6/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-05-27T06:40:02.378 INFO:teuthology.orchestra.run.smithi069.stdout:(7/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 9.2 MB/s | 4.9 MB 00:00 2024-05-27T06:40:02.628 INFO:teuthology.orchestra.run.smithi069.stdout:(8/159): ceph-common-19.0.0-3872.g38f9d900.el9. 11 MB/s | 20 MB 00:01 2024-05-27T06:40:02.720 INFO:teuthology.orchestra.run.smithi069.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 278 kB/s | 25 kB 00:00 2024-05-27T06:40:02.770 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:16 2024-05-27T06:40:03.370 INFO:teuthology.orchestra.run.smithi069.stdout:(10/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 10 MB/s | 10 MB 00:00 2024-05-27T06:40:03.446 INFO:teuthology.orchestra.run.smithi069.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 3.0 MB/s | 224 kB 00:00 2024-05-27T06:40:03.537 INFO:teuthology.orchestra.run.smithi069.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 297 kB/s | 27 kB 00:00 2024-05-27T06:40:03.621 INFO:teuthology.orchestra.run.smithi069.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 384 kB/s | 32 kB 00:00 2024-05-27T06:40:03.721 INFO:teuthology.orchestra.run.smithi069.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 7.4 MB/s | 755 kB 00:00 2024-05-27T06:40:03.789 INFO:teuthology.orchestra.run.smithi069.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 2.3 MB/s | 157 kB 00:00 2024-05-27T06:40:03.864 INFO:teuthology.orchestra.run.smithi069.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.7 MB/s | 127 kB 00:00 2024-05-27T06:40:03.964 INFO:teuthology.orchestra.run.smithi069.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 4.9 MB/s | 498 kB 00:00 2024-05-27T06:40:04.331 INFO:teuthology.orchestra.run.smithi069.stdout:(18/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 8.4 MB/s | 18 MB 00:02 2024-05-27T06:40:04.456 INFO:teuthology.orchestra.run.smithi069.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 365 kB/s | 45 kB 00:00 2024-05-27T06:40:04.584 INFO:teuthology.orchestra.run.smithi069.stdout:(20/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 8.4 MB/s | 5.2 MB 00:00 2024-05-27T06:40:04.615 INFO:teuthology.orchestra.run.smithi069.stdout:(21/159): python3-ceph-common-19.0.0-3872.g38f9 859 kB/s | 136 kB 00:00 2024-05-27T06:40:04.649 INFO:teuthology.orchestra.run.smithi069.stdout:(22/159): python3-cephfs-19.0.0-3872.g38f9d900. 2.5 MB/s | 161 kB 00:00 2024-05-27T06:40:04.691 INFO:teuthology.orchestra.run.smithi069.stdout:(23/159): python3-rados-19.0.0-3872.g38f9d900.e 4.2 MB/s | 320 kB 00:00 2024-05-27T06:40:04.732 INFO:teuthology.orchestra.run.smithi069.stdout:(24/159): python3-rbd-19.0.0-3872.g38f9d900.el9 3.6 MB/s | 300 kB 00:00 2024-05-27T06:40:04.766 INFO:teuthology.orchestra.run.smithi069.stdout:(25/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.3 MB/s | 99 kB 00:00 2024-05-27T06:40:04.792 INFO:teuthology.orchestra.run.smithi069.stdout:(26/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.5 MB/s | 86 kB 00:00 2024-05-27T06:40:04.850 INFO:teuthology.orchestra.run.smithi069.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 2.9 MB/s | 169 kB 00:00 2024-05-27T06:40:04.926 INFO:teuthology.orchestra.run.smithi069.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 460 kB/s | 34 kB 00:00 2024-05-27T06:40:05.009 INFO:teuthology.orchestra.run.smithi069.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.7 MB/s | 143 kB 00:00 2024-05-27T06:40:05.126 INFO:teuthology.orchestra.run.smithi069.stdout:(30/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 8.5 MB/s | 3.0 MB 00:00 2024-05-27T06:40:05.385 INFO:teuthology.orchestra.run.smithi069.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 9.5 MB/s | 3.5 MB 00:00 2024-05-27T06:40:05.468 INFO:teuthology.orchestra.run.smithi069.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 3.3 MB/s | 274 kB 00:00 2024-05-27T06:40:05.552 INFO:teuthology.orchestra.run.smithi069.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 597 kB/s | 49 kB 00:00 2024-05-27T06:40:05.644 INFO:teuthology.orchestra.run.smithi069.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 181 kB/s | 16 kB 00:00 2024-05-27T06:40:05.728 INFO:teuthology.orchestra.run.smithi069.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 3.2 MB/s | 273 kB 00:00 2024-05-27T06:40:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:(36/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 6.4 MB/s | 764 kB 00:00 2024-05-27T06:40:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:(37/159): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-05-27T06:40:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:(38/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 105 kB/s | 18 kB 00:00 2024-05-27T06:40:06.504 INFO:teuthology.orchestra.run.smithi069.stdout:(39/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 11 MB/s | 43 MB 00:03 2024-05-27T06:40:06.529 INFO:teuthology.orchestra.run.smithi069.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 146 kB/s | 80 kB 00:00 2024-05-27T06:40:06.630 INFO:teuthology.orchestra.run.smithi069.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-05-27T06:40:06.663 INFO:teuthology.orchestra.run.smithi069.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-27T06:40:06.697 INFO:teuthology.orchestra.run.smithi069.stdout:(43/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-27T06:40:06.722 INFO:teuthology.orchestra.run.smithi069.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-27T06:40:06.756 INFO:teuthology.orchestra.run.smithi069.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-27T06:40:06.789 INFO:teuthology.orchestra.run.smithi069.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-27T06:40:06.823 INFO:teuthology.orchestra.run.smithi069.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-27T06:40:06.873 INFO:teuthology.orchestra.run.smithi069.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-27T06:40:06.920 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-27T06:40:06.982 INFO:teuthology.orchestra.run.smithi069.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-27T06:40:07.007 INFO:teuthology.orchestra.run.smithi069.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 806 kB/s | 107 kB 00:00 2024-05-27T06:40:07.032 INFO:teuthology.orchestra.run.smithi069.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 439 kB/s | 22 kB 00:00 2024-05-27T06:40:07.116 INFO:teuthology.orchestra.run.smithi069.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-27T06:40:07.141 INFO:teuthology.orchestra.run.smithi069.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-05-27T06:40:07.175 INFO:teuthology.orchestra.run.smithi069.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-05-27T06:40:07.200 INFO:teuthology.orchestra.run.smithi069.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 315 kB/s | 18 kB 00:00 2024-05-27T06:40:07.225 INFO:teuthology.orchestra.run.smithi069.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-05-27T06:40:07.267 INFO:teuthology.orchestra.run.smithi069.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-05-27T06:40:07.293 INFO:teuthology.orchestra.run.smithi069.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 684 kB/s | 45 kB 00:00 2024-05-27T06:40:07.343 INFO:teuthology.orchestra.run.smithi069.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-27T06:40:07.385 INFO:teuthology.orchestra.run.smithi069.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-27T06:40:07.419 INFO:teuthology.orchestra.run.smithi069.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-27T06:40:07.444 INFO:teuthology.orchestra.run.smithi069.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 817 kB/s | 47 kB 00:00 2024-05-27T06:40:07.478 INFO:teuthology.orchestra.run.smithi069.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-27T06:40:07.511 INFO:teuthology.orchestra.run.smithi069.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-27T06:40:07.536 INFO:teuthology.orchestra.run.smithi069.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-05-27T06:40:07.620 INFO:teuthology.orchestra.run.smithi069.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-05-27T06:40:07.771 INFO:teuthology.orchestra.run.smithi069.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T06:40:07.888 INFO:teuthology.orchestra.run.smithi069.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-27T06:40:07.965 INFO:teuthology.orchestra.run.smithi069.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 4.7 MB/s | 911 kB 00:00 2024-05-27T06:40:08.004 INFO:teuthology.orchestra.run.smithi069.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-27T06:40:08.097 INFO:teuthology.orchestra.run.smithi069.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-05-27T06:40:08.138 INFO:teuthology.orchestra.run.smithi069.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-27T06:40:08.189 INFO:teuthology.orchestra.run.smithi069.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-05-27T06:40:08.231 INFO:teuthology.orchestra.run.smithi069.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T06:40:08.281 INFO:teuthology.orchestra.run.smithi069.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-27T06:40:08.331 INFO:teuthology.orchestra.run.smithi069.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T06:40:08.373 INFO:teuthology.orchestra.run.smithi069.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 631 kB/s | 26 kB 00:00 2024-05-27T06:40:08.407 INFO:teuthology.orchestra.run.smithi069.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 588 kB/s | 19 kB 00:00 2024-05-27T06:40:08.524 INFO:teuthology.orchestra.run.smithi069.stdout:(79/159): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-05-27T06:40:08.557 INFO:teuthology.orchestra.run.smithi069.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-27T06:40:08.582 INFO:teuthology.orchestra.run.smithi069.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 603 kB/s | 35 kB 00:00 2024-05-27T06:40:08.633 INFO:teuthology.orchestra.run.smithi069.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-27T06:40:08.775 INFO:teuthology.orchestra.run.smithi069.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-05-27T06:40:08.809 INFO:teuthology.orchestra.run.smithi069.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-05-27T06:40:08.825 INFO:teuthology.orchestra.run.smithi069.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-27T06:40:08.842 INFO:teuthology.orchestra.run.smithi069.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 42 kB 00:00 2024-05-27T06:40:08.876 INFO:teuthology.orchestra.run.smithi069.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-27T06:40:08.910 INFO:teuthology.orchestra.run.smithi069.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-27T06:40:08.943 INFO:teuthology.orchestra.run.smithi069.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-27T06:40:08.969 INFO:teuthology.orchestra.run.smithi069.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 879 kB/s | 51 kB 00:00 2024-05-27T06:40:08.970 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-05-27T06:40:08.994 INFO:teuthology.orchestra.run.smithi069.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T06:40:09.036 INFO:teuthology.orchestra.run.smithi069.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T06:40:09.078 INFO:teuthology.orchestra.run.smithi069.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T06:40:09.128 INFO:teuthology.orchestra.run.smithi069.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-27T06:40:09.237 INFO:teuthology.orchestra.run.smithi069.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.5 MB/s | 5.1 MB 00:00 2024-05-27T06:40:09.262 INFO:teuthology.orchestra.run.smithi069.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-05-27T06:40:09.287 INFO:teuthology.orchestra.run.smithi069.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-05-27T06:40:09.321 INFO:teuthology.orchestra.run.smithi069.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-27T06:40:09.346 INFO:teuthology.orchestra.run.smithi069.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-27T06:40:09.372 INFO:teuthology.orchestra.run.smithi069.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-05-27T06:40:09.397 INFO:teuthology.orchestra.run.smithi069.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T06:40:09.422 INFO:teuthology.orchestra.run.smithi069.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-05-27T06:40:09.483 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:40:09.485 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:40:09.507 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T06:40:09.509 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T06:40:09.510 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T06:40:09.512 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T06:40:09.514 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T06:40:09.518 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T06:40:09.519 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T06:40:09.598 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-27T06:40:09.624 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-27T06:40:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-05-27T06:40:09.624 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-27T06:40:09.624 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-05-27T06:40:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T06:40:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T06:40:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T06:40:09.627 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-05-27T06:40:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T06:40:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T06:40:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T06:40:09.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T06:40:09.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T06:40:09.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T06:40:09.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T06:40:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T06:40:09.636 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 11 Packages 2024-05-27T06:40:09.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:40:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 219 M 2024-05-27T06:40:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-05-27T06:40:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-27T06:40:10.231 INFO:teuthology.orchestra.run.smithi069.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 27 kB/s | 22 kB 00:00 2024-05-27T06:40:10.606 INFO:teuthology.orchestra.run.smithi069.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 610 kB/s | 548 kB 00:00 2024-05-27T06:40:10.631 INFO:teuthology.orchestra.run.smithi069.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 776 kB/s | 309 kB 00:00 2024-05-27T06:40:10.757 INFO:teuthology.orchestra.run.smithi069.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 157 kB/s | 19 kB 00:00 2024-05-27T06:40:10.799 INFO:teuthology.orchestra.run.smithi069.stdout:(108/159): protobuf-compiler-3.14.0-13.el9.x86_ 628 kB/s | 863 kB 00:01 2024-05-27T06:40:10.968 INFO:teuthology.orchestra.run.smithi016.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-27T06:40:11.074 INFO:teuthology.orchestra.run.smithi069.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 83 kB/s | 26 kB 00:00 2024-05-27T06:40:11.124 INFO:teuthology.orchestra.run.smithi069.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 150 kB/s | 49 kB 00:00 2024-05-27T06:40:11.180 INFO:teuthology.orchestra.run.smithi016.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 4.0 MB/s | 863 kB 00:00 2024-05-27T06:40:11.230 INFO:teuthology.orchestra.run.smithi016.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 2.9 MB/s | 147 kB 00:00 2024-05-27T06:40:11.405 INFO:teuthology.orchestra.run.smithi016.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 13 MB/s | 2.3 MB 00:00 2024-05-27T06:40:11.516 INFO:teuthology.orchestra.run.smithi069.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 174 kB/s | 67 kB 00:00 2024-05-27T06:40:11.522 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:40:11.524 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T06:40:11.531 INFO:teuthology.orchestra.run.smithi016.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 12 MB/s | 1.5 MB 00:00 2024-05-27T06:40:11.546 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T06:40:11.547 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T06:40:11.548 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T06:40:11.550 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T06:40:11.552 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T06:40:11.556 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T06:40:11.557 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T06:40:11.616 INFO:teuthology.orchestra.run.smithi069.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-05-27T06:40:11.630 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-05-27T06:40:11.641 INFO:teuthology.orchestra.run.smithi069.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 296 kB/s | 151 kB 00:00 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T06:40:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T06:40:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T06:40:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T06:40:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T06:40:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T06:40:11.655 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T06:40:11.656 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T06:40:11.657 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T06:40:11.658 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T06:40:11.659 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T06:40:11.660 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T06:40:11.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T06:40:11.662 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 11 Packages 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:40:11.663 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 219 M 2024-05-27T06:40:11.664 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-05-27T06:40:11.775 INFO:teuthology.orchestra.run.smithi069.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-05-27T06:40:11.834 INFO:teuthology.orchestra.run.smithi069.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-05-27T06:40:11.839 INFO:teuthology.orchestra.run.smithi016.stdout:(6/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 16 MB/s | 4.9 MB 00:00 2024-05-27T06:40:11.859 INFO:teuthology.orchestra.run.smithi069.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 206 kB/s | 43 kB 00:00 2024-05-27T06:40:11.876 INFO:teuthology.orchestra.run.smithi069.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 334 kB/s | 32 kB 00:00 2024-05-27T06:40:11.959 INFO:teuthology.orchestra.run.smithi069.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-05-27T06:40:11.993 INFO:teuthology.orchestra.run.smithi069.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-27T06:40:12.018 INFO:teuthology.orchestra.run.smithi069.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-27T06:40:12.110 INFO:teuthology.orchestra.run.smithi069.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-27T06:40:12.144 INFO:teuthology.orchestra.run.smithi069.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-27T06:40:12.246 INFO:teuthology.orchestra.run.smithi164.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-27T06:40:12.377 INFO:teuthology.orchestra.run.smithi069.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 47 kB/s | 11 kB 00:00 2024-05-27T06:40:12.436 INFO:teuthology.orchestra.run.smithi069.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-05-27T06:40:12.461 INFO:teuthology.orchestra.run.smithi069.stdout:(125/159): python3-grpcio-tools-1.46.7-10.el9.x 413 kB/s | 144 kB 00:00 2024-05-27T06:40:12.486 INFO:teuthology.orchestra.run.smithi069.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 165 kB/s | 18 kB 00:00 2024-05-27T06:40:12.578 INFO:teuthology.orchestra.run.smithi069.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-05-27T06:40:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 138 kB/s | 19 kB 00:00 2024-05-27T06:40:12.629 INFO:teuthology.orchestra.run.smithi069.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 139 kB/s | 20 kB 00:00 2024-05-27T06:40:12.782 INFO:teuthology.orchestra.run.smithi016.stdout:(7/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-05-27T06:40:12.812 INFO:teuthology.orchestra.run.smithi069.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-05-27T06:40:12.854 INFO:teuthology.orchestra.run.smithi069.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 186 kB/s | 46 kB 00:00 2024-05-27T06:40:12.881 INFO:teuthology.orchestra.run.smithi069.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 315 kB/s | 79 kB 00:00 2024-05-27T06:40:12.897 INFO:teuthology.orchestra.run.smithi164.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 1.3 MB/s | 863 kB 00:00 2024-05-27T06:40:12.947 INFO:teuthology.orchestra.run.smithi069.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 438 kB/s | 58 kB 00:00 2024-05-27T06:40:12.963 INFO:teuthology.orchestra.run.smithi164.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 2.2 MB/s | 147 kB 00:00 2024-05-27T06:40:12.988 INFO:teuthology.orchestra.run.smithi069.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-27T06:40:13.030 INFO:teuthology.orchestra.run.smithi069.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 110 kB/s | 16 kB 00:00 2024-05-27T06:40:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:(8/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x86 15 MB/s | 18 MB 00:01 2024-05-27T06:40:13.056 INFO:teuthology.orchestra.run.smithi069.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-05-27T06:40:13.106 INFO:teuthology.orchestra.run.smithi069.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-05-27T06:40:13.123 INFO:teuthology.orchestra.run.smithi016.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 342 kB/s | 25 kB 00:00 2024-05-27T06:40:13.148 INFO:teuthology.orchestra.run.smithi069.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 266 kB/s | 31 kB 00:00 2024-05-27T06:40:13.181 INFO:teuthology.orchestra.run.smithi069.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-27T06:40:13.223 INFO:teuthology.orchestra.run.smithi069.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-05-27T06:40:13.265 INFO:teuthology.orchestra.run.smithi069.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 311 kB/s | 36 kB 00:00 2024-05-27T06:40:13.349 INFO:teuthology.orchestra.run.smithi069.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 306 kB/s | 51 kB 00:00 2024-05-27T06:40:13.383 INFO:teuthology.orchestra.run.smithi069.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-05-27T06:40:13.408 INFO:teuthology.orchestra.run.smithi069.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 636 kB/s | 90 kB 00:00 2024-05-27T06:40:13.466 INFO:teuthology.orchestra.run.smithi069.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.7 MB/s | 427 kB 00:00 2024-05-27T06:40:13.525 INFO:teuthology.orchestra.run.smithi069.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-05-27T06:40:13.559 INFO:teuthology.orchestra.run.smithi069.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-27T06:40:13.692 INFO:teuthology.orchestra.run.smithi069.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-05-27T06:40:13.901 INFO:teuthology.orchestra.run.smithi069.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-27T06:40:13.993 INFO:teuthology.orchestra.run.smithi069.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-05-27T06:40:14.060 INFO:teuthology.orchestra.run.smithi069.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 631 kB/s | 41 kB 00:00 2024-05-27T06:40:14.097 INFO:teuthology.orchestra.run.smithi164.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-05-27T06:40:14.110 INFO:teuthology.orchestra.run.smithi069.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T06:40:14.177 INFO:teuthology.orchestra.run.smithi069.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 558 kB/s | 37 kB 00:00 2024-05-27T06:40:14.228 INFO:teuthology.orchestra.run.smithi069.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-05-27T06:40:14.278 INFO:teuthology.orchestra.run.smithi069.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.2 MB/s | 260 kB 00:00 2024-05-27T06:40:14.322 INFO:teuthology.orchestra.run.smithi164.stdout:(5/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-27T06:40:14.370 INFO:teuthology.orchestra.run.smithi069.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-05-27T06:40:14.446 INFO:teuthology.orchestra.run.smithi069.stdout:(157/159): librados2-19.0.0-3872.g38f9d900.el9. 3.6 MB/s | 3.3 MB 00:00 2024-05-27T06:40:14.529 INFO:teuthology.orchestra.run.smithi069.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-05-27T06:40:14.672 INFO:teuthology.orchestra.run.smithi164.stdout:(6/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-05-27T06:40:14.741 INFO:teuthology.orchestra.run.smithi016.stdout:(10/159): ceph-common-19.0.0-3872.g38f9d900.el9 4.8 MB/s | 20 MB 00:04 2024-05-27T06:40:14.865 INFO:teuthology.orchestra.run.smithi016.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 1.8 MB/s | 224 kB 00:00 2024-05-27T06:40:14.941 INFO:teuthology.orchestra.run.smithi016.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 363 kB/s | 27 kB 00:00 2024-05-27T06:40:15.016 INFO:teuthology.orchestra.run.smithi016.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 428 kB/s | 32 kB 00:00 2024-05-27T06:40:15.192 INFO:teuthology.orchestra.run.smithi016.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 4.2 MB/s | 755 kB 00:00 2024-05-27T06:40:15.283 INFO:teuthology.orchestra.run.smithi016.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 1.7 MB/s | 157 kB 00:00 2024-05-27T06:40:15.393 INFO:teuthology.orchestra.run.smithi016.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.1 MB/s | 127 kB 00:00 2024-05-27T06:40:15.550 INFO:teuthology.orchestra.run.smithi016.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 3.1 MB/s | 498 kB 00:00 2024-05-27T06:40:15.746 INFO:teuthology.orchestra.run.smithi016.stdout:(18/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 3.4 MB/s | 10 MB 00:02 2024-05-27T06:40:15.781 INFO:teuthology.orchestra.run.smithi164.stdout:(7/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 3.3 MB/s | 4.9 MB 00:01 2024-05-27T06:40:15.818 INFO:teuthology.orchestra.run.smithi016.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 640 kB/s | 45 kB 00:00 2024-05-27T06:40:15.828 INFO:teuthology.orchestra.run.smithi069.stdout:(159/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 1.4 MB/s | 3.1 MB 00:02 2024-05-27T06:40:15.837 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-27T06:40:15.837 INFO:teuthology.orchestra.run.smithi069.stdout:Total 14 MB/s | 219 MB 00:15 2024-05-27T06:40:15.918 INFO:teuthology.orchestra.run.smithi016.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 1.3 MB/s | 136 kB 00:00 2024-05-27T06:40:16.010 INFO:teuthology.orchestra.run.smithi016.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.7 MB/s | 161 kB 00:00 2024-05-27T06:40:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:(22/159): python3-rados-19.0.0-3872.g38f9d900.e 1.7 MB/s | 320 kB 00:00 2024-05-27T06:40:16.394 INFO:teuthology.orchestra.run.smithi016.stdout:(23/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 6.1 MB/s | 5.2 MB 00:00 2024-05-27T06:40:16.452 INFO:teuthology.orchestra.run.smithi016.stdout:(24/159): python3-rbd-19.0.0-3872.g38f9d900.el9 1.1 MB/s | 300 kB 00:00 2024-05-27T06:40:16.478 INFO:teuthology.orchestra.run.smithi016.stdout:(25/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.2 MB/s | 99 kB 00:00 2024-05-27T06:40:16.528 INFO:teuthology.orchestra.run.smithi016.stdout:(26/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-27T06:40:16.620 INFO:teuthology.orchestra.run.smithi016.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 1.8 MB/s | 169 kB 00:00 2024-05-27T06:40:16.712 INFO:teuthology.orchestra.run.smithi016.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 376 kB/s | 34 kB 00:00 2024-05-27T06:40:16.813 INFO:teuthology.orchestra.run.smithi016.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.4 MB/s | 143 kB 00:00 2024-05-27T06:40:17.004 INFO:teuthology.orchestra.run.smithi016.stdout:(30/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 5.8 MB/s | 3.0 MB 00:00 2024-05-27T06:40:17.813 INFO:teuthology.orchestra.run.smithi016.stdout:(31/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 9.2 MB/s | 43 MB 00:04 2024-05-27T06:40:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 2.5 MB/s | 274 kB 00:00 2024-05-27T06:40:17.981 INFO:teuthology.orchestra.run.smithi016.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 861 kB/s | 49 kB 00:00 2024-05-27T06:40:18.064 INFO:teuthology.orchestra.run.smithi016.stdout:(34/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 2.8 MB/s | 3.5 MB 00:01 2024-05-27T06:40:18.072 INFO:teuthology.orchestra.run.smithi164.stdout:(8/159): ceph-radosgw-19.0.0-3872.g38f9d900.el9 4.4 MB/s | 10 MB 00:02 2024-05-27T06:40:18.089 INFO:teuthology.orchestra.run.smithi016.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3872.g3 153 kB/s | 16 kB 00:00 2024-05-27T06:40:18.123 INFO:teuthology.orchestra.run.smithi164.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 517 kB/s | 25 kB 00:00 2024-05-27T06:40:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-05-27T06:40:18.257 INFO:teuthology.orchestra.run.smithi016.stdout:(37/159): ceph-volume-19.0.0-3872.g38f9d900.el9 1.4 MB/s | 273 kB 00:00 2024-05-27T06:40:18.307 INFO:teuthology.orchestra.run.smithi016.stdout:(38/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 3.4 MB/s | 764 kB 00:00 2024-05-27T06:40:18.332 INFO:teuthology.orchestra.run.smithi016.stdout:(39/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 169 kB/s | 18 kB 00:00 2024-05-27T06:40:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-05-27T06:40:18.466 INFO:teuthology.orchestra.run.smithi016.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-27T06:40:18.500 INFO:teuthology.orchestra.run.smithi016.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-27T06:40:18.542 INFO:teuthology.orchestra.run.smithi016.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-27T06:40:18.584 INFO:teuthology.orchestra.run.smithi016.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-27T06:40:18.632 INFO:teuthology.orchestra.run.smithi164.stdout:(10/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 4.5 MB/s | 18 MB 00:03 2024-05-27T06:40:18.637 INFO:teuthology.orchestra.run.smithi016.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-27T06:40:18.668 INFO:teuthology.orchestra.run.smithi016.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-27T06:40:18.709 INFO:teuthology.orchestra.run.smithi016.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-27T06:40:18.715 INFO:teuthology.orchestra.run.smithi164.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 2.7 MB/s | 224 kB 00:00 2024-05-27T06:40:18.743 INFO:teuthology.orchestra.run.smithi016.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 215 kB 00:00 2024-05-27T06:40:18.787 INFO:teuthology.orchestra.run.smithi164.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 552 kB/s | 27 kB 00:00 2024-05-27T06:40:18.999 INFO:teuthology.orchestra.run.smithi164.stdout:(13/159): ceph-common-19.0.0-3872.g38f9d900.el9 2.8 MB/s | 20 MB 00:07 2024-05-27T06:40:19.007 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-05-27T06:40:19.016 INFO:teuthology.orchestra.run.smithi164.stdout:(14/159): libcephfs-devel-19.0.0-3872.g38f9d900 127 kB/s | 32 kB 00:00 2024-05-27T06:40:19.066 INFO:teuthology.orchestra.run.smithi164.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 3.1 MB/s | 157 kB 00:00 2024-05-27T06:40:19.150 INFO:teuthology.orchestra.run.smithi164.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.5 MB/s | 127 kB 00:00 2024-05-27T06:40:19.209 INFO:teuthology.orchestra.run.smithi164.stdout:(17/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 3.5 MB/s | 755 kB 00:00 2024-05-27T06:40:19.252 INFO:teuthology.orchestra.run.smithi016.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 210 kB/s | 107 kB 00:00 2024-05-27T06:40:19.259 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-05-27T06:40:19.259 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-05-27T06:40:19.293 INFO:teuthology.orchestra.run.smithi164.stdout:(18/159): libradosstriper1-19.0.0-3872.g38f9d90 3.4 MB/s | 498 kB 00:00 2024-05-27T06:40:19.343 INFO:teuthology.orchestra.run.smithi164.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 931 kB/s | 45 kB 00:00 2024-05-27T06:40:19.361 INFO:teuthology.orchestra.run.smithi016.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 201 kB/s | 22 kB 00:00 2024-05-27T06:40:19.401 INFO:teuthology.orchestra.run.smithi164.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 2.3 MB/s | 136 kB 00:00 2024-05-27T06:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 2.7 MB/s | 161 kB 00:00 2024-05-27T06:40:19.461 INFO:teuthology.orchestra.run.smithi016.stdout:(51/159): automake-1.16.2-8.el9.noarch.rpm 870 kB/s | 689 kB 00:00 2024-05-27T06:40:19.511 INFO:teuthology.orchestra.run.smithi016.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-27T06:40:19.544 INFO:teuthology.orchestra.run.smithi164.stdout:(22/159): python3-rados-19.0.0-3872.g38f9d900.e 3.8 MB/s | 320 kB 00:00 2024-05-27T06:40:19.561 INFO:teuthology.orchestra.run.smithi016.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 330 kB/s | 33 kB 00:00 2024-05-27T06:40:19.619 INFO:teuthology.orchestra.run.smithi164.stdout:(23/159): python3-rbd-19.0.0-3872.g38f9d900.el9 4.0 MB/s | 300 kB 00:00 2024-05-27T06:40:19.670 INFO:teuthology.orchestra.run.smithi016.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-27T06:40:19.677 INFO:teuthology.orchestra.run.smithi164.stdout:(24/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.7 MB/s | 99 kB 00:00 2024-05-27T06:40:19.720 INFO:teuthology.orchestra.run.smithi016.stdout:(55/159): flex-2.6.4-9.el9.x86_64.rpm 875 kB/s | 313 kB 00:00 2024-05-27T06:40:19.736 INFO:teuthology.orchestra.run.smithi164.stdout:(25/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.5 MB/s | 86 kB 00:00 2024-05-27T06:40:19.779 INFO:teuthology.orchestra.run.smithi016.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-27T06:40:19.829 INFO:teuthology.orchestra.run.smithi016.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-27T06:40:19.888 INFO:teuthology.orchestra.run.smithi016.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-05-27T06:40:20.045 INFO:teuthology.orchestra.run.smithi164.stdout:(26/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 6.2 MB/s | 5.2 MB 00:00 2024-05-27T06:40:20.095 INFO:teuthology.orchestra.run.smithi164.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 3.3 MB/s | 169 kB 00:00 2024-05-27T06:40:20.138 INFO:teuthology.orchestra.run.smithi016.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-05-27T06:40:20.146 INFO:teuthology.orchestra.run.smithi164.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 695 kB/s | 34 kB 00:00 2024-05-27T06:40:20.180 INFO:teuthology.orchestra.run.smithi016.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-05-27T06:40:20.213 INFO:teuthology.orchestra.run.smithi164.stdout:(29/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 6.4 MB/s | 3.0 MB 00:00 2024-05-27T06:40:20.250 INFO:teuthology.orchestra.run.smithi016.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-27T06:40:20.271 INFO:teuthology.orchestra.run.smithi164.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.1 MB/s | 143 kB 00:00 2024-05-27T06:40:20.289 INFO:teuthology.orchestra.run.smithi016.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-05-27T06:40:20.365 INFO:teuthology.orchestra.run.smithi016.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-05-27T06:40:20.398 INFO:teuthology.orchestra.run.smithi016.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-27T06:40:20.474 INFO:teuthology.orchestra.run.smithi016.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-05-27T06:40:20.699 INFO:teuthology.orchestra.run.smithi016.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-05-27T06:40:20.705 INFO:teuthology.orchestra.run.smithi164.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 7.2 MB/s | 3.5 MB 00:00 2024-05-27T06:40:20.790 INFO:teuthology.orchestra.run.smithi164.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 3.2 MB/s | 274 kB 00:00 2024-05-27T06:40:20.848 INFO:teuthology.orchestra.run.smithi164.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 855 kB/s | 49 kB 00:00 2024-05-27T06:40:20.899 INFO:teuthology.orchestra.run.smithi164.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 332 kB/s | 16 kB 00:00 2024-05-27T06:40:20.983 INFO:teuthology.orchestra.run.smithi164.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 3.2 MB/s | 273 kB 00:00 2024-05-27T06:40:21.126 INFO:teuthology.orchestra.run.smithi164.stdout:(36/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 5.3 MB/s | 764 kB 00:00 2024-05-27T06:40:21.185 INFO:teuthology.orchestra.run.smithi164.stdout:(37/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 318 kB/s | 18 kB 00:00 2024-05-27T06:40:21.335 INFO:teuthology.orchestra.run.smithi164.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-05-27T06:40:21.694 INFO:teuthology.orchestra.run.smithi164.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 159 kB/s | 80 kB 00:00 2024-05-27T06:40:21.865 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-05-27T06:40:21.867 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-05-27T06:40:21.961 INFO:teuthology.orchestra.run.smithi164.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 734 kB/s | 195 kB 00:00 2024-05-27T06:40:22.074 INFO:teuthology.orchestra.run.smithi016.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-05-27T06:40:22.178 INFO:teuthology.orchestra.run.smithi164.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-27T06:40:22.228 INFO:teuthology.orchestra.run.smithi164.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 901 kB/s | 803 kB 00:00 2024-05-27T06:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 990 kB/s | 106 kB 00:00 2024-05-27T06:40:22.453 INFO:teuthology.orchestra.run.smithi164.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-27T06:40:22.572 INFO:teuthology.orchestra.run.smithi164.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-05-27T06:40:22.604 INFO:teuthology.orchestra.run.smithi164.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 835 kB/s | 125 kB 00:00 2024-05-27T06:40:22.687 INFO:teuthology.orchestra.run.smithi164.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-05-27T06:40:22.838 INFO:teuthology.orchestra.run.smithi164.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-05-27T06:40:22.857 INFO:teuthology.orchestra.run.smithi016.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 754 kB/s | 1.6 MB 00:02 2024-05-27T06:40:22.880 INFO:teuthology.orchestra.run.smithi164.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-27T06:40:23.124 INFO:teuthology.orchestra.run.smithi016.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 501 kB/s | 133 kB 00:00 2024-05-27T06:40:23.324 INFO:teuthology.orchestra.run.smithi016.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 729 kB/s | 911 kB 00:01 2024-05-27T06:40:23.355 INFO:teuthology.orchestra.run.smithi164.stdout:(50/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 8.3 MB/s | 43 MB 00:05 2024-05-27T06:40:23.380 INFO:teuthology.orchestra.run.smithi164.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 43 kB/s | 22 kB 00:00 2024-05-27T06:40:23.430 INFO:teuthology.orchestra.run.smithi164.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-05-27T06:40:23.456 INFO:teuthology.orchestra.run.smithi164.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 439 kB/s | 33 kB 00:00 2024-05-27T06:40:23.506 INFO:teuthology.orchestra.run.smithi164.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 370 kB/s | 18 kB 00:00 2024-05-27T06:40:23.540 INFO:teuthology.orchestra.run.smithi164.stdout:(55/159): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-05-27T06:40:23.565 INFO:teuthology.orchestra.run.smithi164.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-05-27T06:40:23.598 INFO:teuthology.orchestra.run.smithi164.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-27T06:40:23.615 INFO:teuthology.orchestra.run.smithi164.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 918 kB/s | 45 kB 00:00 2024-05-27T06:40:23.749 INFO:teuthology.orchestra.run.smithi164.stdout:(59/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-05-27T06:40:23.783 INFO:teuthology.orchestra.run.smithi164.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-05-27T06:40:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-05-27T06:40:23.833 INFO:teuthology.orchestra.run.smithi164.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-05-27T06:40:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-05-27T06:40:23.875 INFO:teuthology.orchestra.run.smithi016.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.4 MB/s | 4.7 MB 00:03 2024-05-27T06:40:23.892 INFO:teuthology.orchestra.run.smithi164.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-27T06:40:23.918 INFO:teuthology.orchestra.run.smithi164.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 35 kB 00:00 2024-05-27T06:40:23.951 INFO:teuthology.orchestra.run.smithi164.stdout:(66/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-27T06:40:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 479 kB/s | 87 kB 00:00 2024-05-27T06:40:24.060 INFO:teuthology.orchestra.run.smithi164.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-27T06:40:24.177 INFO:teuthology.orchestra.run.smithi164.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-27T06:40:24.277 INFO:teuthology.orchestra.run.smithi164.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-27T06:40:24.319 INFO:teuthology.orchestra.run.smithi164.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-05-27T06:40:24.353 INFO:teuthology.orchestra.run.smithi164.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 764 kB/s | 133 kB 00:00 2024-05-27T06:40:24.403 INFO:teuthology.orchestra.run.smithi164.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-27T06:40:24.445 INFO:teuthology.orchestra.run.smithi164.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-05-27T06:40:24.512 INFO:teuthology.orchestra.run.smithi164.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 929 kB/s | 61 kB 00:00 2024-05-27T06:40:24.587 INFO:teuthology.orchestra.run.smithi164.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-27T06:40:24.646 INFO:teuthology.orchestra.run.smithi164.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-27T06:40:24.788 INFO:teuthology.orchestra.run.smithi164.stdout:(77/159): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-05-27T06:40:24.813 INFO:teuthology.orchestra.run.smithi164.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 154 kB/s | 26 kB 00:00 2024-05-27T06:40:24.884 INFO:teuthology.orchestra.run.smithi016.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 663 kB/s | 1.0 MB 00:01 2024-05-27T06:40:24.947 INFO:teuthology.orchestra.run.smithi164.stdout:(79/159): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-27T06:40:24.972 INFO:teuthology.orchestra.run.smithi164.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 19 kB 00:00 2024-05-27T06:40:24.997 INFO:teuthology.orchestra.run.smithi164.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 706 kB/s | 35 kB 00:00 2024-05-27T06:40:25.001 INFO:teuthology.orchestra.run.smithi016.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 529 kB/s | 61 kB 00:00 2024-05-27T06:40:25.031 INFO:teuthology.orchestra.run.smithi164.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 796 kB/s | 172 kB 00:00 2024-05-27T06:40:25.064 INFO:teuthology.orchestra.run.smithi164.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-27T06:40:25.106 INFO:teuthology.orchestra.run.smithi164.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-27T06:40:25.132 INFO:teuthology.orchestra.run.smithi164.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-27T06:40:25.157 INFO:teuthology.orchestra.run.smithi164.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 855 kB/s | 42 kB 00:00 2024-05-27T06:40:25.190 INFO:teuthology.orchestra.run.smithi164.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-27T06:40:25.249 INFO:teuthology.orchestra.run.smithi164.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-27T06:40:25.283 INFO:teuthology.orchestra.run.smithi164.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-27T06:40:25.308 INFO:teuthology.orchestra.run.smithi164.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-05-27T06:40:25.333 INFO:teuthology.orchestra.run.smithi164.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T06:40:25.359 INFO:teuthology.orchestra.run.smithi016.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 683 kB/s | 244 kB 00:00 2024-05-27T06:40:25.376 INFO:teuthology.orchestra.run.smithi164.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T06:40:25.418 INFO:teuthology.orchestra.run.smithi164.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T06:40:25.468 INFO:teuthology.orchestra.run.smithi164.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-27T06:40:25.577 INFO:teuthology.orchestra.run.smithi164.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.4 MB/s | 5.1 MB 00:00 2024-05-27T06:40:25.627 INFO:teuthology.orchestra.run.smithi164.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 227 kB/s | 36 kB 00:00 2024-05-27T06:40:25.644 INFO:teuthology.orchestra.run.smithi164.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 718 kB/s | 47 kB 00:00 2024-05-27T06:40:25.690 INFO:teuthology.orchestra.run.smithi164.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-27T06:40:25.711 INFO:teuthology.orchestra.run.smithi164.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 871 kB/s | 58 kB 00:00 2024-05-27T06:40:25.731 INFO:teuthology.orchestra.run.smithi016.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 669 kB/s | 248 kB 00:00 2024-05-27T06:40:25.744 INFO:teuthology.orchestra.run.smithi164.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-05-27T06:40:25.770 INFO:teuthology.orchestra.run.smithi164.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-27T06:40:25.795 INFO:teuthology.orchestra.run.smithi164.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-05-27T06:40:25.843 INFO:teuthology.orchestra.run.smithi016.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 231 kB/s | 26 kB 00:00 2024-05-27T06:40:25.862 INFO:teuthology.orchestra.run.smithi164.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-05-27T06:40:25.954 INFO:teuthology.orchestra.run.smithi164.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-05-27T06:40:25.960 INFO:teuthology.orchestra.run.smithi016.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 165 kB/s | 19 kB 00:00 2024-05-27T06:40:26.179 INFO:teuthology.orchestra.run.smithi164.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-27T06:40:26.235 INFO:teuthology.orchestra.run.smithi016.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 627 kB/s | 172 kB 00:00 2024-05-27T06:40:26.344 INFO:teuthology.orchestra.run.smithi016.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 323 kB/s | 35 kB 00:00 2024-05-27T06:40:26.439 INFO:teuthology.orchestra.run.smithi164.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-05-27T06:40:26.538 INFO:teuthology.orchestra.run.smithi164.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 812 kB/s | 548 kB 00:00 2024-05-27T06:40:26.622 INFO:teuthology.orchestra.run.smithi164.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 463 kB/s | 309 kB 00:00 2024-05-27T06:40:26.647 INFO:teuthology.orchestra.run.smithi164.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-05-27T06:40:26.764 INFO:teuthology.orchestra.run.smithi164.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 344 kB/s | 49 kB 00:00 2024-05-27T06:40:26.789 INFO:teuthology.orchestra.run.smithi164.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-05-27T06:40:26.881 INFO:teuthology.orchestra.run.smithi164.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-27T06:40:27.057 INFO:teuthology.orchestra.run.smithi164.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-05-27T06:40:27.182 INFO:teuthology.orchestra.run.smithi164.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 350 kB/s | 43 kB 00:00 2024-05-27T06:40:27.241 INFO:teuthology.orchestra.run.smithi164.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-05-27T06:40:27.291 INFO:teuthology.orchestra.run.smithi164.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 298 kB/s | 32 kB 00:00 2024-05-27T06:40:27.366 INFO:teuthology.orchestra.run.smithi164.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-27T06:40:27.433 INFO:teuthology.orchestra.run.smithi164.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-05-27T06:40:27.575 INFO:teuthology.orchestra.run.smithi164.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-05-27T06:40:27.609 INFO:teuthology.orchestra.run.smithi164.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-05-27T06:40:27.692 INFO:teuthology.orchestra.run.smithi164.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-27T06:40:27.834 INFO:teuthology.orchestra.run.smithi164.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-27T06:40:27.985 INFO:teuthology.orchestra.run.smithi164.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-05-27T06:40:28.010 INFO:teuthology.orchestra.run.smithi164.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 61 kB/s | 11 kB 00:00 2024-05-27T06:40:28.144 INFO:teuthology.orchestra.run.smithi164.stdout:(125/159): libarrow-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-05-27T06:40:28.169 INFO:teuthology.orchestra.run.smithi164.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 96 kB/s | 18 kB 00:00 2024-05-27T06:40:28.194 INFO:teuthology.orchestra.run.smithi164.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-05-27T06:40:28.269 INFO:teuthology.orchestra.run.smithi164.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-05-27T06:40:28.294 INFO:teuthology.orchestra.run.smithi164.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-05-27T06:40:28.378 INFO:teuthology.orchestra.run.smithi164.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 430 kB/s | 46 kB 00:00 2024-05-27T06:40:28.412 INFO:teuthology.orchestra.run.smithi164.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 679 kB/s | 79 kB 00:00 2024-05-27T06:40:28.448 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-05-27T06:40:28.462 INFO:teuthology.orchestra.run.smithi164.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-05-27T06:40:28.487 INFO:teuthology.orchestra.run.smithi164.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2024-05-27T06:40:28.537 INFO:teuthology.orchestra.run.smithi164.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-27T06:40:28.543 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T06:40:28.571 INFO:teuthology.orchestra.run.smithi164.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-27T06:40:28.605 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T06:40:28.625 INFO:teuthology.orchestra.run.smithi164.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 658 kB/s | 90 kB 00:00 2024-05-27T06:40:28.655 INFO:teuthology.orchestra.run.smithi164.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-05-27T06:40:28.680 INFO:teuthology.orchestra.run.smithi164.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-05-27T06:40:28.715 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T06:40:28.739 INFO:teuthology.orchestra.run.smithi164.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-27T06:40:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 547 kB/s | 59 kB 00:00 2024-05-27T06:40:28.770 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T06:40:28.789 INFO:teuthology.orchestra.run.smithi164.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-05-27T06:40:28.803 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T06:40:28.848 INFO:teuthology.orchestra.run.smithi164.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 472 kB/s | 51 kB 00:00 2024-05-27T06:40:28.881 INFO:teuthology.orchestra.run.smithi164.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-27T06:40:28.902 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T06:40:28.907 INFO:teuthology.orchestra.run.smithi164.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 769 kB/s | 90 kB 00:00 2024-05-27T06:40:29.015 INFO:teuthology.orchestra.run.smithi164.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-05-27T06:40:29.041 INFO:teuthology.orchestra.run.smithi164.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 126 kB/s | 20 kB 00:00 2024-05-27T06:40:29.074 INFO:teuthology.orchestra.run.smithi164.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-27T06:40:29.275 INFO:teuthology.orchestra.run.smithi164.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-05-27T06:40:29.332 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T06:40:29.433 INFO:teuthology.orchestra.run.smithi164.stdout:(149/159): librados2-19.0.0-3872.g38f9d900.el9. 8.5 MB/s | 3.3 MB 00:00 2024-05-27T06:40:29.435 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:40:29.600 INFO:teuthology.orchestra.run.smithi164.stdout:(150/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-05-27T06:40:29.617 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:40:29.644 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:40:29.770 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:40:29.802 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:40:29.887 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T06:40:29.951 INFO:teuthology.orchestra.run.smithi164.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 119 kB/s | 41 kB 00:00 2024-05-27T06:40:30.226 INFO:teuthology.orchestra.run.smithi164.stdout:(152/159): NetworkManager-1.47.90-1.el9.x86_64. 2.4 MB/s | 2.3 MB 00:00 2024-05-27T06:40:30.230 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T06:40:30.308 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T06:40:30.326 INFO:teuthology.orchestra.run.smithi164.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 370 kB/s | 37 kB 00:00 2024-05-27T06:40:30.342 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:40:30.351 INFO:teuthology.orchestra.run.smithi164.stdout:(154/159): NetworkManager-tui-1.47.90-1.el9.x86 620 kB/s | 248 kB 00:00 2024-05-27T06:40:30.443 INFO:teuthology.orchestra.run.smithi164.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-05-27T06:40:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:40:30.534 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:40:30.560 INFO:teuthology.orchestra.run.smithi164.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.2 MB/s | 260 kB 00:00 2024-05-27T06:40:30.620 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:40:30.652 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:40:30.686 INFO:teuthology.orchestra.run.smithi164.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-05-27T06:40:30.719 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:40:30.895 INFO:teuthology.orchestra.run.smithi016.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 762 kB/s | 5.8 MB 00:07 2024-05-27T06:40:31.044 INFO:teuthology.orchestra.run.smithi164.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 2.4 MB 00:00 2024-05-27T06:40:31.150 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:40:31.177 INFO:teuthology.orchestra.run.smithi164.stdout:(159/159): NetworkManager-libnm-1.47.90-1.el9.x 1.0 MB/s | 1.8 MB 00:01 2024-05-27T06:40:31.184 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-05-27T06:40:31.184 INFO:teuthology.orchestra.run.smithi164.stdout:Total 11 MB/s | 219 MB 00:19 2024-05-27T06:40:31.245 INFO:teuthology.orchestra.run.smithi016.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 719 kB/s | 250 kB 00:00 2024-05-27T06:40:31.536 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:40:31.595 INFO:teuthology.orchestra.run.smithi016.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 636 kB/s | 222 kB 00:00 2024-05-27T06:40:31.750 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:40:31.787 INFO:teuthology.orchestra.run.smithi016.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 404 kB/s | 77 kB 00:00 2024-05-27T06:40:31.839 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:40:31.896 INFO:teuthology.orchestra.run.smithi016.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 391 kB/s | 42 kB 00:00 2024-05-27T06:40:32.005 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:32.153 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:32.188 INFO:teuthology.orchestra.run.smithi016.stdout:(86/159): python3-babel-2.9.1-2.el9.noarch.rpm 751 kB/s | 6.0 MB 00:08 2024-05-27T06:40:32.388 INFO:teuthology.orchestra.run.smithi016.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 548 kB/s | 269 kB 00:00 2024-05-27T06:40:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 597 kB/s | 159 kB 00:00 2024-05-27T06:40:32.469 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 473 kB/s | 51 kB 00:00 2024-05-27T06:40:32.657 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T06:40:32.672 INFO:teuthology.orchestra.run.smithi016.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 499 kB/s | 54 kB 00:00 2024-05-27T06:40:32.692 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T06:40:32.745 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T06:40:32.853 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T06:40:32.855 INFO:teuthology.orchestra.run.smithi016.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 598 kB/s | 279 kB 00:00 2024-05-27T06:40:32.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T06:40:32.965 INFO:teuthology.orchestra.run.smithi016.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-05-27T06:40:32.977 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T06:40:33.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T06:40:33.057 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T06:40:33.073 INFO:teuthology.orchestra.run.smithi016.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-05-27T06:40:33.160 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T06:40:33.224 INFO:teuthology.orchestra.run.smithi016.stdout:(94/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 759 kB/s | 5.1 MB 00:06 2024-05-27T06:40:33.235 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T06:40:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T06:40:33.328 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T06:40:33.332 INFO:teuthology.orchestra.run.smithi016.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 334 kB/s | 36 kB 00:00 2024-05-27T06:40:33.438 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T06:40:33.441 INFO:teuthology.orchestra.run.smithi016.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-05-27T06:40:33.466 INFO:teuthology.orchestra.run.smithi016.stdout:(97/159): qatlib-24.02.0-1.el9.x86_64.rpm 563 kB/s | 221 kB 00:00 2024-05-27T06:40:33.478 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T06:40:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:40:33.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T06:40:33.575 INFO:teuthology.orchestra.run.smithi016.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 537 kB/s | 58 kB 00:00 2024-05-27T06:40:33.675 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T06:40:33.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T06:40:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 491 kB/s | 94 kB 00:00 2024-05-27T06:40:33.865 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T06:40:33.892 INFO:teuthology.orchestra.run.smithi016.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 677 kB/s | 305 kB 00:00 2024-05-27T06:40:33.951 INFO:teuthology.orchestra.run.smithi016.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-05-27T06:40:34.001 INFO:teuthology.orchestra.run.smithi016.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 415 kB/s | 45 kB 00:00 2024-05-27T06:40:34.040 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T06:40:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-05-27T06:40:34.108 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T06:40:34.343 INFO:teuthology.orchestra.run.smithi016.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-27T06:40:34.410 INFO:teuthology.orchestra.run.smithi016.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-05-27T06:40:34.415 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-05-27T06:40:34.460 INFO:teuthology.orchestra.run.smithi016.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2024-05-27T06:40:34.661 INFO:teuthology.orchestra.run.smithi016.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-27T06:40:34.671 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-05-27T06:40:34.671 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-05-27T06:40:34.719 INFO:teuthology.orchestra.run.smithi016.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 453 kB/s | 26 kB 00:00 2024-05-27T06:40:34.778 INFO:teuthology.orchestra.run.smithi016.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 840 kB/s | 49 kB 00:00 2024-05-27T06:40:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T06:40:34.837 INFO:teuthology.orchestra.run.smithi016.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-27T06:40:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T06:40:34.904 INFO:teuthology.orchestra.run.smithi016.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-27T06:40:34.925 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T06:40:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T06:40:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T06:40:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T06:40:35.078 INFO:teuthology.orchestra.run.smithi016.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-27T06:40:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T06:40:35.231 INFO:teuthology.orchestra.run.smithi016.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-05-27T06:40:35.271 INFO:teuthology.orchestra.run.smithi016.stdout:(114/159): protobuf-compiler-3.14.0-13.el9.x86_ 680 kB/s | 863 kB 00:01 2024-05-27T06:40:35.297 INFO:teuthology.orchestra.run.smithi016.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 673 kB/s | 43 kB 00:00 2024-05-27T06:40:35.355 INFO:teuthology.orchestra.run.smithi016.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-05-27T06:40:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T06:40:35.398 INFO:teuthology.orchestra.run.smithi016.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-05-27T06:40:35.423 INFO:teuthology.orchestra.run.smithi016.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-27T06:40:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T06:40:35.490 INFO:teuthology.orchestra.run.smithi016.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-27T06:40:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T06:40:35.582 INFO:teuthology.orchestra.run.smithi016.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-27T06:40:35.615 INFO:teuthology.orchestra.run.smithi016.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-05-27T06:40:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T06:40:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T06:40:35.715 INFO:teuthology.orchestra.run.smithi016.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-27T06:40:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-27T06:40:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T06:40:35.774 INFO:teuthology.orchestra.run.smithi016.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-05-27T06:40:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T06:40:35.808 INFO:teuthology.orchestra.run.smithi016.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 305 kB/s | 18 kB 00:00 2024-05-27T06:40:35.833 INFO:teuthology.orchestra.run.smithi016.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 399 kB/s | 23 kB 00:00 2024-05-27T06:40:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T06:40:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-05-27T06:40:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T06:40:35.913 INFO:teuthology.orchestra.run.smithi016.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 249 kB/s | 20 kB 00:00 2024-05-27T06:40:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T06:40:35.968 INFO:teuthology.orchestra.run.smithi016.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-05-27T06:40:35.993 INFO:teuthology.orchestra.run.smithi016.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 590 kB/s | 46 kB 00:00 2024-05-27T06:40:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T06:40:36.026 INFO:teuthology.orchestra.run.smithi016.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-27T06:40:36.052 INFO:teuthology.orchestra.run.smithi016.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 997 kB/s | 58 kB 00:00 2024-05-27T06:40:36.085 INFO:teuthology.orchestra.run.smithi016.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-05-27T06:40:36.111 INFO:teuthology.orchestra.run.smithi016.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-05-27T06:40:36.144 INFO:teuthology.orchestra.run.smithi016.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-27T06:40:36.161 INFO:teuthology.orchestra.run.smithi016.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 569 kB/s | 28 kB 00:00 2024-05-27T06:40:36.195 INFO:teuthology.orchestra.run.smithi016.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-05-27T06:40:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T06:40:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T06:40:36.259 INFO:teuthology.orchestra.run.smithi016.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-05-27T06:40:36.296 INFO:teuthology.orchestra.run.smithi016.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-05-27T06:40:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T06:40:36.313 INFO:teuthology.orchestra.run.smithi016.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 679 kB/s | 36 kB 00:00 2024-05-27T06:40:36.355 INFO:teuthology.orchestra.run.smithi016.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 883 kB/s | 51 kB 00:00 2024-05-27T06:40:36.388 INFO:teuthology.orchestra.run.smithi016.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-27T06:40:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-27T06:40:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T06:40:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T06:40:36.472 INFO:teuthology.orchestra.run.smithi016.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-05-27T06:40:36.506 INFO:teuthology.orchestra.run.smithi016.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-05-27T06:40:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T06:40:36.539 INFO:teuthology.orchestra.run.smithi016.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-27T06:40:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T06:40:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T06:40:36.648 INFO:teuthology.orchestra.run.smithi016.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T06:40:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T06:40:36.857 INFO:teuthology.orchestra.run.smithi016.stdout:(148/159): librados2-19.0.0-3872.g38f9d900.el9. 11 MB/s | 3.3 MB 00:00 2024-05-27T06:40:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T06:40:36.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T06:40:37.048 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T06:40:37.094 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T06:40:37.115 INFO:teuthology.orchestra.run.smithi016.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 9.0 MB/s | 2.3 MB 00:00 2024-05-27T06:40:37.162 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T06:40:37.223 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T06:40:37.224 INFO:teuthology.orchestra.run.smithi016.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-05-27T06:40:37.279 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T06:40:37.291 INFO:teuthology.orchestra.run.smithi016.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 631 kB/s | 41 kB 00:00 2024-05-27T06:40:37.326 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T06:40:37.341 INFO:teuthology.orchestra.run.smithi016.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T06:40:37.373 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T06:40:37.391 INFO:teuthology.orchestra.run.smithi016.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 751 kB/s | 37 kB 00:00 2024-05-27T06:40:37.442 INFO:teuthology.orchestra.run.smithi016.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-05-27T06:40:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T06:40:37.454 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-05-27T06:40:37.455 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-05-27T06:40:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T06:40:37.517 INFO:teuthology.orchestra.run.smithi016.stdout:(155/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-27T06:40:37.550 INFO:teuthology.orchestra.run.smithi016.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.4 MB/s | 260 kB 00:00 2024-05-27T06:40:37.641 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T06:40:37.676 INFO:teuthology.orchestra.run.smithi016.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-27T06:40:37.700 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T06:40:37.742 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T06:40:37.743 INFO:teuthology.orchestra.run.smithi016.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-05-27T06:40:37.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T06:40:37.959 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T06:40:38.033 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T06:40:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T06:40:39.040 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T06:40:39.073 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T06:40:39.193 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T06:40:39.264 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T06:40:39.318 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T06:40:39.419 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T06:40:39.459 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T06:40:39.684 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T06:40:39.714 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T06:40:39.814 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T06:40:40.674 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T06:40:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T06:40:42.430 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T06:40:42.532 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T06:40:42.565 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T06:40:42.601 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T06:40:42.979 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T06:40:43.034 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:40:43.190 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:40:43.224 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T06:40:43.291 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T06:40:43.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T06:40:43.887 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:40:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:40:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-05-27T06:40:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T06:40:44.407 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T06:40:44.475 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T06:40:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T06:40:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T06:40:44.703 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T06:40:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T06:40:45.229 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T06:40:45.360 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:40:45.540 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:40:45.568 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:40:45.678 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:40:45.750 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:40:45.841 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T06:40:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T06:40:46.279 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T06:40:46.298 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:40:46.458 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:40:46.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:40:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:40:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:40:46.697 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:40:46.909 INFO:teuthology.orchestra.run.smithi016.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.1 MB/s | 16 MB 00:14 2024-05-27T06:40:46.915 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-27T06:40:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:Total 5.9 MB/s | 219 MB 00:37 2024-05-27T06:40:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:40:47.070 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:40:47.110 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:40:47.410 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:40:47.644 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:40:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:40:47.790 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:40:47.854 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:40:47.925 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:48.076 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:40:48.076 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T06:40:48.077 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T06:40:48.077 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:40:48.085 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:48.123 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:40:48.423 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:40:48.635 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T06:40:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T06:40:48.721 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T06:40:48.864 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T06:40:48.921 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T06:40:48.986 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T06:40:49.032 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T06:40:49.073 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T06:40:49.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T06:40:49.260 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T06:40:49.304 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T06:40:49.371 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T06:40:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T06:40:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T06:40:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:40:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T06:40:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T06:40:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T06:40:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T06:40:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T06:40:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T06:40:50.266 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-05-27T06:40:50.536 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-05-27T06:40:50.536 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-05-27T06:40:50.895 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T06:40:50.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T06:40:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T06:40:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T06:40:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T06:40:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T06:40:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T06:40:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T06:40:51.606 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T06:40:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T06:40:51.818 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T06:40:51.914 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T06:40:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T06:40:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T06:40:52.069 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T06:40:52.109 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T06:40:52.170 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T06:40:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T06:40:52.462 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T06:40:52.521 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T06:40:52.599 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T06:40:52.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T06:40:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T06:40:52.852 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T06:40:52.887 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T06:40:52.919 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T06:40:53.149 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T06:40:53.151 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-05-27T06:40:53.152 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-05-27T06:40:53.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T06:40:53.374 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T06:40:53.414 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T06:40:53.464 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T06:40:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T06:40:53.580 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T06:40:53.628 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T06:40:53.673 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T06:40:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T06:40:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T06:40:53.872 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T06:40:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T06:40:54.108 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T06:40:54.164 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T06:40:54.237 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T06:40:54.392 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T06:40:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T06:40:55.444 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T06:40:55.500 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T06:40:55.533 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T06:40:55.656 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T06:40:55.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T06:40:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T06:40:55.872 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T06:40:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T06:40:56.139 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T06:40:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T06:40:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T06:40:57.124 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T06:40:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T06:40:58.930 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T06:40:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T06:40:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T06:40:59.136 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T06:40:59.517 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T06:40:59.559 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:40:59.671 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:40:59.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T06:40:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T06:41:00.134 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-05-27T06:41:00.224 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T06:41:00.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T06:41:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T06:41:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:41:00.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T06:41:00.495 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:41:00.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T06:41:00.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T06:41:00.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T06:41:00.648 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T06:41:00.829 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:41:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-05-27T06:41:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-05-27T06:41:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-05-27T06:41:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-05-27T06:41:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-05-27T06:41:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-05-27T06:41:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-05-27T06:41:00.830 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:01.056 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:01.102 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:01.103 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:01.103 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T06:41:01.103 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T06:41:01.215 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:41:01.407 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:41:01.441 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:41:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:41:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:41:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T06:41:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T06:41:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T06:41:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:41:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:41:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:41:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T06:41:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:41:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T06:41:02.781 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:02.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:41:03.131 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:03.206 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:03.207 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:03.207 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T06:41:03.207 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:03.207 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:03.207 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:03.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:03.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T06:41:03.582 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:41:03.678 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T06:41:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:03.779 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:03.779 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T06:41:03.780 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:03.780 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:03.893 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:41:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T06:41:03.955 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:41:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T06:41:04.034 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:41:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T06:41:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:41:04.107 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T06:41:04.107 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T06:41:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:41:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T06:41:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T06:41:04.378 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T06:41:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T06:41:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T06:41:04.588 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T06:41:04.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T06:41:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T06:41:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T06:41:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T06:41:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T06:41:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T06:41:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T06:41:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T06:41:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T06:41:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T06:41:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T06:41:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T06:41:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T06:41:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T06:41:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T06:41:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T06:41:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T06:41:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:41:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T06:41:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:05.757 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T06:41:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T06:41:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T06:41:05.901 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T06:41:06.146 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T06:41:06.215 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T06:41:06.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T06:41:07.059 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T06:41:07.106 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T06:41:07.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T06:41:07.191 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:07.202 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T06:41:07.236 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:07.250 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T06:41:07.322 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T06:41:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T06:41:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T06:41:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T06:41:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T06:41:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T06:41:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T06:41:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T06:41:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T06:41:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T06:41:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T06:41:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T06:41:08.327 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:08.361 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:08.518 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T06:41:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T06:41:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T06:41:08.706 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T06:41:08.738 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:08.818 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T06:41:08.826 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:08.826 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:08.827 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T06:41:08.827 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:08.827 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:08.827 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:08.858 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T06:41:08.897 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:08.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T06:41:08.935 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:08.940 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T06:41:08.963 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T06:41:09.071 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T06:41:09.135 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T06:41:09.155 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T06:41:09.338 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T06:41:09.396 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T06:41:09.436 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T06:41:09.480 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T06:41:09.542 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T06:41:09.592 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T06:41:09.635 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T06:41:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T06:41:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T06:41:09.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T06:41:09.845 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T06:41:09.996 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T06:41:10.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T06:41:10.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T06:41:10.186 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T06:41:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T06:41:10.225 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:10.317 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:10.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T06:41:10.367 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:10.434 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T06:41:10.464 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T06:41:10.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:10.551 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:10.642 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:10.642 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:10.642 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T06:41:10.642 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:10.642 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:10.643 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:11.052 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:11.149 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:11.149 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:11.149 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T06:41:11.150 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:11.150 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:11.150 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:11.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T06:41:11.499 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T06:41:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T06:41:11.670 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T06:41:11.767 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T06:41:11.840 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T06:41:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T06:41:11.994 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T06:41:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T06:41:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T06:41:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T06:41:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T06:41:13.425 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T06:41:15.141 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T06:41:15.246 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T06:41:15.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T06:41:15.316 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T06:41:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T06:41:15.732 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:41:15.857 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T06:41:15.890 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T06:41:15.914 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T06:41:15.923 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T06:41:15.968 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T06:41:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T06:41:16.006 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T06:41:16.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T06:41:16.366 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T06:41:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T06:41:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T06:41:16.485 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T06:41:16.535 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T06:41:16.552 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T06:41:16.594 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:41:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:41:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T06:41:16.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T06:41:16.997 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:41:16.997 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-05-27T06:41:16.998 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:17.277 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:17.277 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T06:41:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:17.660 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:17.763 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:19.056 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:19.138 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:19.138 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T06:41:19.138 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:19.138 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T06:41:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T06:41:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T06:41:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T06:41:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T06:41:19.465 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:19.465 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:19.465 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T06:41:19.465 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:19.465 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:19.466 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:19.491 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:41:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T06:41:19.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T06:41:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T06:41:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T06:41:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:20.069 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:20.199 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T06:41:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T06:41:20.275 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:20.276 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:20.284 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T06:41:20.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:41:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T06:41:20.457 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:20.498 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T06:41:20.504 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T06:41:20.504 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T06:41:20.505 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T06:41:20.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:20.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:41:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T06:41:20.617 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:20.643 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T06:41:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T06:41:20.846 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T06:41:20.922 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:20.922 INFO:teuthology.orchestra.run.smithi069.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T06:41:20.922 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T06:41:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T06:41:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T06:41:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T06:41:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:21.879 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:21.935 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T06:41:22.045 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T06:41:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:23.417 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:23.457 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:24.557 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:24.583 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:24.928 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T06:41:24.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:25.048 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:25.048 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:25.048 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T06:41:25.048 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:25.049 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:25.049 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:25.147 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T06:41:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T06:41:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T06:41:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:26.469 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:26.469 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T06:41:26.470 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:26.470 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:26.620 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:26.620 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T06:41:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:26.740 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T06:41:26.740 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T06:41:26.740 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:41:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T06:41:26.742 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T06:41:26.743 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T06:41:26.743 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T06:41:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T06:41:26.756 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T06:41:26.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T06:41:26.758 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T06:41:26.759 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T06:41:26.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T06:41:26.761 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T06:41:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T06:41:26.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T06:41:26.764 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T06:41:26.765 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T06:41:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T06:41:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T06:41:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T06:41:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T06:41:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:27.140 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T06:41:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.975 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.976 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.977 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T06:41:28.978 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T06:41:28.980 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T06:41:28.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T06:41:28.983 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T06:41:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T06:41:28.986 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T06:41:28.987 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T06:41:28.987 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T06:41:28.987 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-27T06:41:28.987 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-27T06:41:29.628 DEBUG:teuthology.parallel:result is None 2024-05-27T06:41:31.888 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T06:41:31.943 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T06:41:31.989 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T06:41:32.026 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T06:41:32.061 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T06:41:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T06:41:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T06:41:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T06:41:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T06:41:32.502 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T06:41:32.606 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:41:33.517 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:33.587 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:33.657 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-05-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:34.496 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:34.548 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T06:41:34.548 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:34.548 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T06:41:34.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:35.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T06:41:35.152 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T06:41:35.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T06:41:35.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T06:41:35.421 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T06:41:35.445 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:41:35.480 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T06:41:35.520 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T06:41:35.520 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T06:41:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:35.863 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:35.902 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T06:41:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:36.348 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:36.422 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T06:41:36.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:36.463 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T06:41:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:36.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:36.781 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T06:41:36.781 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:36.781 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T06:41:36.782 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:36.782 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T06:41:36.782 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:36.845 INFO:teuthology.orchestra.run.smithi164.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T06:41:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:37.315 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:37.387 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T06:41:37.387 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:37.388 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T06:41:37.388 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:37.388 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T06:41:37.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:37.533 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T06:41:37.606 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T06:41:37.684 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T06:41:37.934 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T06:41:38.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T06:41:38.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T06:41:38.211 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T06:41:38.389 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T06:41:38.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T06:41:38.517 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T06:41:38.551 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T06:41:39.212 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:39.250 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T06:41:39.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T06:41:39.408 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T06:41:39.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:40.807 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T06:41:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:42.011 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T06:41:42.046 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:42.214 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T06:41:42.408 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T06:41:42.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T06:41:42.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:42.590 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:42.630 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T06:41:42.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T06:41:42.814 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:41:42.851 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T06:41:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T06:41:42.853 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T06:41:42.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T06:41:42.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T06:41:42.856 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T06:41:42.857 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T06:41:42.858 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T06:41:42.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T06:41:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T06:41:42.863 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T06:41:42.864 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T06:41:42.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T06:41:42.866 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T06:41:43.917 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T06:41:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T06:41:44.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:44.119 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T06:41:44.195 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:44.195 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T06:41:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T06:41:44.376 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:44.377 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T06:41:44.377 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:44.377 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T06:41:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T06:41:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T06:41:45.184 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.185 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.186 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T06:41:45.187 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T06:41:45.188 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T06:41:45.189 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T06:41:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T06:41:45.191 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T06:41:45.192 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T06:41:45.193 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T06:41:45.194 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T06:41:45.195 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T06:41:45.196 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T06:41:45.197 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T06:41:45.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:41:45.197 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-05-27T06:41:45.859 DEBUG:teuthology.parallel:result is None 2024-05-27T06:41:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T06:41:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T06:41:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T06:41:50.000 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T06:41:50.041 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T06:41:50.350 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T06:41:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T06:41:50.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T06:41:50.479 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T06:41:50.526 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T06:41:50.580 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:41:51.576 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:51.685 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T06:41:53.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T06:41:53.268 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T06:41:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T06:41:53.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T06:41:53.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T06:41:53.595 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:41:53.629 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T06:41:53.671 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T06:41:53.672 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:53.829 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T06:41:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T06:41:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:54.047 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T06:41:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T06:41:54.118 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:54.119 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:54.472 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T06:41:54.472 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:54.618 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T06:41:54.661 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T06:41:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T06:41:55.105 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T06:41:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:42:00.947 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T06:42:00.948 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T06:42:00.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T06:42:00.962 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T06:42:00.962 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T06:42:00.962 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T06:42:00.962 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T06:42:00.962 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T06:42:00.963 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T06:42:00.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T06:42:00.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T06:42:00.966 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T06:42:00.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T06:42:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T06:42:00.969 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T06:42:00.970 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T06:42:00.971 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T06:42:00.972 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T06:42:00.973 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T06:42:00.973 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T06:42:03.476 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T06:42:03.476 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:42:03.476 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-05-27T06:42:03.476 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T06:42:03.476 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T06:42:03.477 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.478 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T06:42:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T06:42:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T06:42:03.482 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T06:42:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T06:42:03.484 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T06:42:03.485 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T06:42:03.486 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T06:42:03.487 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T06:42:03.488 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T06:42:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T06:42:03.490 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T06:42:03.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:42:03.491 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-27T06:42:04.100 DEBUG:teuthology.parallel:result is None 2024-05-27T06:42:04.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:04.252 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T06:42:04.303 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.304 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.304 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T06:42:04.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:04.449 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T06:42:04.523 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.523 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.523 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T06:42:04.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:04.747 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T06:42:04.821 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.821 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T06:42:04.821 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T06:42:04.823 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T06:42:04.823 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:42:04.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T06:42:04.896 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:42:04.896 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T06:42:04.989 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:42:04.989 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T06:42:05.082 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T06:42:05.082 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:42:05.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T06:42:05.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T06:42:05.248 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:42:05.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T06:42:05.338 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T06:42:05.466 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:42:05.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T06:42:05.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T06:42:05.678 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T06:42:05.678 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:42:05.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T06:42:05.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T06:42:05.847 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:42:05.847 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T06:42:05.928 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T06:42:06.059 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:42:06.059 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T06:42:06.146 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T06:42:06.273 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T06:42:06.274 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:42:06.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T06:42:06.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T06:42:06.457 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:42:06.457 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T06:42:06.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T06:42:06.632 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:42:06.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T06:42:06.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T06:42:06.804 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T06:42:06.898 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'subvols': {'create': 2}} 2024-05-27T06:42:06.899 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:06.899 INFO:tasks.cephadm:Cluster fsid is 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:42:06.899 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T06:42:06.899 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.69', 'mon.c': '172.21.15.164'} 2024-05-27T06:42:06.900 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-05-27T06:42:06.900 INFO:tasks.cephadm:First mgr is x 2024-05-27T06:42:06.900 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T06:42:06.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-05-27T06:42:06.971 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-05-27T06:42:07.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-05-27T06:42:07.152 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-27T06:42:07.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:07.302 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79587/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 08:08:15.155937', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79583/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 07:54:08.730060', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-27T06:42:07.445 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240521.063558-debug, sha1 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:42:07.446 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T06:42:07.446 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T06:42:07.447 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:08.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 06:42 /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:08.125 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:08.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 06:42 /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:08.863 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:09.433 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 06:42 /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:09.434 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:09.498 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:09.556 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T06:42:09.628 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 on all hosts... 2024-05-27T06:42:09.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T06:42:09.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T06:42:09.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T06:42:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T06:42:10.829 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T06:42:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T06:42:54.823 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-27T06:42:54.824 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-27T06:43:00.165 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-27T06:43:00.165 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T06:43:00.165 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T06:43:00.165 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-05-27T06:43:00.166 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T06:43:00.166 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-27T06:43:00.166 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-27T06:43:01.552 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:43:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T06:43:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T06:43:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-05-27T06:43:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T06:43:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T06:43:01.553 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:43:01.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-05-27T06:43:01.705 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-05-27T06:43:01.797 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-05-27T06:43:01.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-05-27T06:43:01.982 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-05-27T06:43:02.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-05-27T06:43:02.153 INFO:tasks.cephadm:Writing seed config... 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T06:43:02.155 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-27T06:43:02.156 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T06:43:02.157 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-27T06:43:02.158 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-27T06:43:02.159 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:43:02.159 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T06:43:02.219 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3c4673a4-1bf4-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T06:43:02.219 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service 2024-05-27T06:43:02.267 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x.service 2024-05-27T06:43:02.312 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T06:43:02.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 -v bootstrap --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T06:43:02.587 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-27T06:43:02.588 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6', '-v', 'bootstrap', '--fsid', '3c4673a4-1bf4-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-05-27T06:43:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T06:43:02.588 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-05-27T06:43:02.647 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.0.2 2024-05-27T06:43:02.647 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-05-27T06:43:02.647 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-05-27T06:43:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T06:43:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T06:43:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T06:43:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:43:02.702 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-05-27T06:43:02.726 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-05-27T06:43:02.726 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-05-27T06:43:02.726 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-05-27T06:43:02.790 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.0.2 2024-05-27T06:43:02.790 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T06:43:02.790 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-05-27T06:43:02.790 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-05-27T06:43:02.808 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T06:43:02.808 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T06:43:02.827 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T06:43:02.828 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:43:02.855 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-05-27T06:43:02.878 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-05-27T06:43:02.878 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-05-27T06:43:02.878 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-05-27T06:43:02.879 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:02.879 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 139699939443472 on /run/cephadm/3c4673a4-1bf4-11ef-bc9b-c7b262605968.lock 2024-05-27T06:43:02.880 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 139699939443472 acquired on /run/cephadm/3c4673a4-1bf4-11ef-bc9b-c7b262605968.lock 2024-05-27T06:43:02.881 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-05-27T06:43:02.882 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-05-27T06:43:02.883 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-05-27T06:43:02.898 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-05-27T06:43:02.899 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-05-27T06:43:02.911 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T06:43:02.912 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-27T06:43:02.912 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-27T06:43:02.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T06:43:02.924 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T06:43:02.924 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T06:43:02.925 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-27T06:43:02.925 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-05-27T06:43:02.925 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T06:43:02.927 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-05-27T06:43:02.928 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-05-27T06:43:02.928 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T06:43:02.931 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T06:43:02.934 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:cf46015854c6cdc81d3273f73f8f001fe4a1edc6ed77e464da8c9d5e01e9b108 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-27T06:43:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T06:43:03.622 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T06:43:04.944 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T06:43:04.944 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T06:43:04.944 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T06:43:06.147 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-05-27T06:43:06.147 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-05-27T06:43:07.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB6K1Rm5uQDKhAABpGaeYpuLJ1AEhIFKbuoLQ== 2024-05-27T06:43:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB7K1RmHATKMxAAZLiBvpNzTbwD4N0yI6wm7Q== 2024-05-27T06:43:09.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB9K1Rm5ePTBxAAc4l/O3lFzTAI//Wyh4oysw== 2024-05-27T06:43:09.768 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T06:43:11.002 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:43:11.003 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-05-27T06:43:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-05-27T06:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T06:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T06:43:13.176 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target. 2024-05-27T06:43:13.176 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target. 2024-05-27T06:43:13.631 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a 2024-05-27T06:43:13.631 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-05-27T06:43:13.936 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target.wants/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@.service. 2024-05-27T06:43:14.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 podman[31018]: 2024-05-27 06:43:14.358590598 +0000 UTC m=+0.137723800 container create 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T06:43:14.774 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T06:43:14.775 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-27T06:43:14.798 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T06:43:14.798 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:43:14.798 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-27T06:43:14.798 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T06:43:14.799 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-05-27T06:43:14.799 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-05-27T06:43:14.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 podman[31018]: 2024-05-27 06:43:14.648362515 +0000 UTC m=+0.427495723 container init 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T06:43:14.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 podman[31018]: 2024-05-27 06:43:14.659287991 +0000 UTC m=+0.438421177 container start 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T06:43:14.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 bash[31018]: 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e 2024-05-27T06:43:14.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 systemd[1]: Started Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:43:14.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 ceph-mon[31036]: mkfs 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:14.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:14 smithi016 ceph-mon[31036]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T06:43:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: monmap epoch 1 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: last_changed 2024-05-27T06:43:10.303844+0000 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: min_mon_release 19 (squid) 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: election_strategy: 1 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: fsmap 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: mgrmap e1: no daemons active 2024-05-27T06:43:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:15 smithi016 ceph-mon[31036]: from='client.? 172.21.15.16:0/2554621720' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T06:43:16.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T06:43:16.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.762519s) 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:16.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.486+0000 7f3790000700 1 Processor -- start 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.487+0000 7f3790000700 1 -- start start 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.487+0000 7f3790000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:16.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.487+0000 7f3790000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f378807d780 con 0x7f378807ee10 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.489+0000 7f378d600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.489+0000 7f378d600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50228/0 (socket says 172.21.15.16:50228) 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.489+0000 7f378d600700 1 -- 172.21.15.16:0/3798100645 learned_addr learned my addr 172.21.15.16:0/3798100645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.489+0000 7f378d600700 1 -- 172.21.15.16:0/3798100645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378807d900 con 0x7f378807ee10 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.490+0000 7f378d600700 1 --2- 172.21.15.16:0/3798100645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f377c0098a0 tx=0x7f377c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c1a4330900f268 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.490+0000 7f3787e00700 1 -- 172.21.15.16:0/3798100645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f377c004160 con 0x7f378807ee10 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.490+0000 7f3787e00700 1 -- 172.21.15.16:0/3798100645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377c004300 con 0x7f378807ee10 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.490+0000 7f3787e00700 1 -- 172.21.15.16:0/3798100645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f377c0036a0 con 0x7f378807ee10 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 -- 172.21.15.16:0/3798100645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 msgr2=0x7f378807d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 --2- 172.21.15.16:0/3798100645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f377c0098a0 tx=0x7f377c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:16.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 -- 172.21.15.16:0/3798100645 shutdown_connections 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 --2- 172.21.15.16:0/3798100645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378807d240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 -- 172.21.15.16:0/3798100645 >> 172.21.15.16:0/3798100645 conn(0x7f37881050b0 msgr2=0x7f3788107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 -- 172.21.15.16:0/3798100645 shutdown_connections 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.491+0000 7f3790000700 1 -- 172.21.15.16:0/3798100645 wait complete. 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.493+0000 7f3790000700 1 Processor -- start 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.493+0000 7f3790000700 1 -- start start 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.493+0000 7f3790000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.493+0000 7f3790000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f378807de30 con 0x7f378807ee10 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.493+0000 7f378d600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.494+0000 7f378d600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50244/0 (socket says 172.21.15.16:50244) 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.494+0000 7f378d600700 1 -- 172.21.15.16:0/2554621720 learned_addr learned my addr 172.21.15.16:0/2554621720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.494+0000 7f378d600700 1 -- 172.21.15.16:0/2554621720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378819fce0 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.494+0000 7f378d600700 1 --2- 172.21.15.16:0/2554621720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f377c000c00 tx=0x7f377c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f377c003db0 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377c011630 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f377c011950 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378819ff70 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881a2b80 con 0x7f378807ee10 2024-05-27T06:43:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f377c011af0 con 0x7f378807ee10 2024-05-27T06:43:16.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.495+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f377c01c450 con 0x7f378807ee10 2024-05-27T06:43:16.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.496+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3788064e10 con 0x7f378807ee10 2024-05-27T06:43:16.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.500+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f377c032020 con 0x7f378807ee10 2024-05-27T06:43:16.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.580+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f378802e8e0 con 0x7f378807ee10 2024-05-27T06:43:16.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.581+0000 7f3786000700 1 -- 172.21.15.16:0/2554621720 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f377c004260 con 0x7f378807ee10 2024-05-27T06:43:16.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.582+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 msgr2=0x7f378819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:16.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.582+0000 7f3790000700 1 --2- 172.21.15.16:0/2554621720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f377c000c00 tx=0x7f377c004750 comp rx=0 tx=0).stop 2024-05-27T06:43:16.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.582+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 shutdown_connections 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.582+0000 7f3790000700 1 --2- 172.21.15.16:0/2554621720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378807ee10 0x7f378819f7a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.582+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 >> 172.21.15.16:0/2554621720 conn(0x7f37881050b0 msgr2=0x7f3788105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.583+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 shutdown_connections 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:15.583+0000 7f3790000700 1 -- 172.21.15.16:0/2554621720 wait complete. 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-05-27T06:43:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T06:43:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:17 smithi016 ceph-mon[31036]: from='client.? 172.21.15.16:0/2640102610' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T06:43:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:17 smithi016 ceph-mon[31036]: from='client.? 172.21.15.16:0/2640102610' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-05-27T06:43:17.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T06:43:17.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T06:43:17.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T06:43:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T06:43:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T06:43:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T06:43:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.975+0000 7f9b81400700 1 Processor -- start 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.976+0000 7f9b81400700 1 -- start start 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.976+0000 7f9b81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.976+0000 7f9b81400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c07d670 con 0x7f9b7c1083a0 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.976+0000 7f9b7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.977+0000 7f9b7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50260/0 (socket says 172.21.15.16:50260) 2024-05-27T06:43:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.977+0000 7f9b7a800700 1 -- 172.21.15.16:0/645020373 learned_addr learned my addr 172.21.15.16:0/645020373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.977+0000 7f9b7a800700 1 -- 172.21.15.16:0/645020373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c108cf0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.977+0000 7f9b7a800700 1 --2- 172.21.15.16:0/645020373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098a0 tx=0x7f9b6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe7254000e22f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.978+0000 7f9b79400700 1 -- 172.21.15.16:0/645020373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c004160 con 0x7f9b7c1083a0 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.978+0000 7f9b79400700 1 -- 172.21.15.16:0/645020373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c004300 con 0x7f9b7c1083a0 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.978+0000 7f9b79400700 1 -- 172.21.15.16:0/645020373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c0036a0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 -- 172.21.15.16:0/645020373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 msgr2=0x7f9b7c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 --2- 172.21.15.16:0/645020373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098a0 tx=0x7f9b6c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 -- 172.21.15.16:0/645020373 shutdown_connections 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 --2- 172.21.15.16:0/645020373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c1087b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 -- 172.21.15.16:0/645020373 >> 172.21.15.16:0/645020373 conn(0x7f9b7c103f90 msgr2=0x7f9b7c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 -- 172.21.15.16:0/645020373 shutdown_connections 2024-05-27T06:43:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.979+0000 7f9b81400700 1 -- 172.21.15.16:0/645020373 wait complete. 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.981+0000 7f9b81400700 1 Processor -- start 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.981+0000 7f9b81400700 1 -- start start 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.981+0000 7f9b81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.981+0000 7f9b81400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1090a0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.982+0000 7f9b7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.982+0000 7f9b7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50266/0 (socket says 172.21.15.16:50266) 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.982+0000 7f9b7a800700 1 -- 172.21.15.16:0/2640102610 learned_addr learned my addr 172.21.15.16:0/2640102610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.982+0000 7f9b7a800700 1 -- 172.21.15.16:0/2640102610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c19fcb0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.982+0000 7f9b7a800700 1 --2- 172.21.15.16:0/2640102610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c000c00 tx=0x7f9b6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c003db0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c011630 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c011950 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c19ff40 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c1a2b50 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.983+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b6c011af0 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.984+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b6c01c450 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.984+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c064e10 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:16.987+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9b6c032020 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.066+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9b7c1a2d20 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.072+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f9b6c004260 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.073+0000 7f9b73400700 1 -- 172.21.15.16:0/2640102610 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9b6c01c650 con 0x7f9b7c1083a0 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 msgr2=0x7f9b7c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:17.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 --2- 172.21.15.16:0/2640102610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c000c00 tx=0x7f9b6c004750 comp rx=0 tx=0).stop 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 shutdown_connections 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 --2- 172.21.15.16:0/2640102610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b7c1083a0 0x7f9b7c19f770 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 >> 172.21.15.16:0/2640102610 conn(0x7f9b7c103f90 msgr2=0x7f9b7c195a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 shutdown_connections 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:17.074+0000 7f9b81400700 1 -- 172.21.15.16:0/2640102610 wait complete. 2024-05-27T06:43:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-05-27T06:43:18.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:18 smithi016 ceph-mon[31036]: from='client.? 172.21.15.16:0/2061289573' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:43:30.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.462+0000 7f073ae00700 1 Processor -- start 2024-05-27T06:43:30.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.462+0000 7f073ae00700 1 -- start start 2024-05-27T06:43:30.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.462+0000 7f073ae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:30.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.462+0000 7f073ae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0734113200 con 0x7f07341107f0 2024-05-27T06:43:30.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.463+0000 7f0733e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:30.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.463+0000 7f0733e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50274/0 (socket says 172.21.15.16:50274) 2024-05-27T06:43:30.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.463+0000 7f0733e00700 1 -- 172.21.15.16:0/4124894061 learned_addr learned my addr 172.21.15.16:0/4124894061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:30.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.463+0000 7f0733e00700 1 -- 172.21.15.16:0/4124894061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0734113380 con 0x7f07341107f0 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.464+0000 7f0733e00700 1 --2- 172.21.15.16:0/4124894061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e80 tx=0x7f072400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6441f2b83a460045 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.464+0000 7f0733400700 1 -- 172.21.15.16:0/4124894061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0724017070 con 0x7f07341107f0 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.464+0000 7f0733400700 1 -- 172.21.15.16:0/4124894061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f072400b8c0 con 0x7f07341107f0 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.464+0000 7f0733400700 1 -- 172.21.15.16:0/4124894061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0724004420 con 0x7f07341107f0 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.465+0000 7f073ae00700 1 -- 172.21.15.16:0/4124894061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 msgr2=0x7f0734112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.465+0000 7f073ae00700 1 --2- 172.21.15.16:0/4124894061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e80 tx=0x7f072400b330 comp rx=0 tx=0).stop 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.465+0000 7f073ae00700 1 -- 172.21.15.16:0/4124894061 shutdown_connections 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.465+0000 7f073ae00700 1 --2- 172.21.15.16:0/4124894061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f0734112c10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.465+0000 7f073ae00700 1 -- 172.21.15.16:0/4124894061 >> 172.21.15.16:0/4124894061 conn(0x7f073407f0b0 msgr2=0x7f0734081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.466+0000 7f073ae00700 1 -- 172.21.15.16:0/4124894061 shutdown_connections 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.466+0000 7f073ae00700 1 -- 172.21.15.16:0/4124894061 wait complete. 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f073ae00700 1 Processor -- start 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f073ae00700 1 -- start start 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f073ae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f073ae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07341143a0 con 0x7f07341107f0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f0733e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f0733e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50282/0 (socket says 172.21.15.16:50282) 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.467+0000 7f0733e00700 1 -- 172.21.15.16:0/2061289573 learned_addr learned my addr 172.21.15.16:0/2061289573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.468+0000 7f0733e00700 1 -- 172.21.15.16:0/2061289573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07341a48c0 con 0x7f07341107f0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.468+0000 7f0733e00700 1 --2- 172.21.15.16:0/2061289573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0724004000 tx=0x7f0724003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0724020020 con 0x7f07341107f0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f072400be50 con 0x7f07341107f0 2024-05-27T06:43:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0724017040 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07341a4b50 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07341a7760 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.469+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0724023020 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.470+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f072400f050 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.470+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0734064e10 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.473+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f072402a020 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.552+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f07341a7930 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.552+0000 7f0731600700 1 -- 172.21.15.16:0/2061289573 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f072400b860 con 0x7f07341107f0 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 msgr2=0x7f07341a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 --2- 172.21.15.16:0/2061289573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0724004000 tx=0x7f0724003d00 comp rx=0 tx=0).stop 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 shutdown_connections 2024-05-27T06:43:30.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 --2- 172.21.15.16:0/2061289573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07341107f0 0x7f07341a4380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 >> 172.21.15.16:0/2061289573 conn(0x7f073407f0b0 msgr2=0x7f073407f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 shutdown_connections 2024-05-27T06:43:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:18.554+0000 7f073ae00700 1 -- 172.21.15.16:0/2061289573 wait complete. 2024-05-27T06:43:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-05-27T06:43:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:30 smithi016 systemd[1]: Stopping Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:43:31.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a[31032]: 2024-05-27T06:43:30.990+0000 7f6a52a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T06:43:31.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a[31032]: 2024-05-27T06:43:30.990+0000 7f6a52a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T06:43:31.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:31 smithi016 podman[31280]: 2024-05-27 06:43:31.084640462 +0000 UTC m=+0.218661187 container died 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T06:43:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:31 smithi016 podman[31280]: 2024-05-27 06:43:31.508587544 +0000 UTC m=+0.642608247 container cleanup 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T06:43:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:31 smithi016 bash[31280]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a 2024-05-27T06:43:31.980 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:31 smithi016 podman[31291]: 2024-05-27 06:43:31.718199337 +0000 UTC m=+0.618320019 container remove 5ddec2f4aa37c3f41016e5678d015a0a4538a8adce61af052a4d00d334cf226e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T06:43:32.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-27T06:43:32.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 systemd[1]: Stopped Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:43:32.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service: Consumed 2.072s CPU time. 2024-05-27T06:43:32.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 systemd[1]: Starting Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:43:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 podman[31397]: 2024-05-27 06:43:32.456151508 +0000 UTC m=+0.043753720 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:43:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 podman[31397]: 2024-05-27 06:43:32.552172927 +0000 UTC m=+0.139775154 container create 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T06:43:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 podman[31397]: 2024-05-27 06:43:32.813893971 +0000 UTC m=+0.401496227 container init 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T06:43:32.904 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 podman[31397]: 2024-05-27 06:43:32.825755821 +0000 UTC m=+0.413358043 container start 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 bash[31397]: 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 systemd[1]: Started Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: pidfile_write: ignore empty --pid-file 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: load: jerasure load: lrc 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: RocksDB version: 7.9.2 2024-05-27T06:43:33.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Git sha 0 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: DB SUMMARY 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: DB Session ID: E6DXWKD8TWVX71G5RUEI 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: CURRENT file: CURRENT 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104572 ; 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.error_if_exists: 0 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.create_if_missing: 0 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.paranoid_checks: 1 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T06:43:33.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.env: 0x55aa5f792500 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.info_log: 0x55aa63252040 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.statistics: (nil) 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.use_fsync: 0 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_log_file_size: 0 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T06:43:33.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_fallocate: 1 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.use_direct_reads: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.db_log_dir: 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.wal_dir: 2024-05-27T06:43:33.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.write_buffer_manager: 0x55aa6311c8c0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T06:43:33.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.unordered_write: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.row_cache: None 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.wal_filter: None 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.two_write_queues: 0 2024-05-27T06:43:33.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.wal_compression: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.atomic_flush: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.log_readahead_size: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T06:43:33.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_background_jobs: 2 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_background_compactions: -1 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_subcompactions: 1 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_open_files: -1 2024-05-27T06:43:33.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_background_flushes: -1 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Compression algorithms supported: 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kZSTD supported: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kXpressCompression supported: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kZlibCompression supported: 1 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kSnappyCompression supported: 1 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kLZ4Compression supported: 1 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: kBZip2Compression supported: 0 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T06:43:33.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.merge_operator: 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_filter: None 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aa63252200) 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T06:43:33.077 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x55aa6314f1f0 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-05-27T06:43:33.078 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-05-27T06:43:33.079 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression: NoCompression 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T06:43:33.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.num_levels: 7 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T06:43:33.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T06:43:33.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T06:43:33.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T06:43:33.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.inplace_update_support: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.bloom_locality: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.max_successive_merges: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T06:43:33.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.ttl: 2592000 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enable_blob_files: false 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.min_blob_size: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T06:43:33.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1039fcc0-90e9-42e0-b666-356b899c70cc 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792212958953, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792212960993, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99811, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97693, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14036, "raw_average_key_size": 49, "raw_value_size": 90031, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716792212, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1039fcc0-90e9-42e0-b666-356b899c70cc", "db_session_id": "E6DXWKD8TWVX71G5RUEI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792212961142, "job": 1, "event": "recovery_finished"} 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aa63133c00 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: DB pointer 0x55aa67f26000 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T06:43:33.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 99.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 99.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:43:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 13.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:43:33.089 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 13.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x55aa6314f1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???) e1 preinit fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).mds e1 new map 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).mds e1 print_map 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T06:43:33.090 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:32 smithi016 ceph-mon[31415]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: monmap epoch 1 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: last_changed 2024-05-27T06:43:10.303844+0000 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: min_mon_release 19 (squid) 2024-05-27T06:43:33.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: election_strategy: 1 2024-05-27T06:43:33.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:43:33.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: fsmap 2024-05-27T06:43:33.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T06:43:33.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:33 smithi016 ceph-mon[31415]: mgrmap e1: no daemons active 2024-05-27T06:43:34.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.768+0000 7fa5ede00700 1 Processor -- start 2024-05-27T06:43:34.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.768+0000 7fa5ede00700 1 -- start start 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.768+0000 7fa5ede00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.768+0000 7fa5ede00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810c470 con 0x7fa5e810ba70 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.769+0000 7fa5eca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.769+0000 7fa5eca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33810/0 (socket says 172.21.15.16:33810) 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.769+0000 7fa5eca00700 1 -- 172.21.15.16:0/2377661191 learned_addr learned my addr 172.21.15.16:0/2377661191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.769+0000 7fa5eca00700 1 -- 172.21.15.16:0/2377661191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e810cc40 con 0x7fa5e810ba70 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.770+0000 7fa5eca00700 1 --2- 172.21.15.16:0/2377661191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00098a0 tx=0x7fa5d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8a1726a7c6ca00 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.770+0000 7fa5e7e00700 1 -- 172.21.15.16:0/2377661191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d0004160 con 0x7fa5e810ba70 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.770+0000 7fa5e7e00700 1 -- 172.21.15.16:0/2377661191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5d0004300 con 0x7fa5e810ba70 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.770+0000 7fa5e7e00700 1 -- 172.21.15.16:0/2377661191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d00119e0 con 0x7fa5e810ba70 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 -- 172.21.15.16:0/2377661191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 msgr2=0x7fa5e810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 --2- 172.21.15.16:0/2377661191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00098a0 tx=0x7fa5d0009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 -- 172.21.15.16:0/2377661191 shutdown_connections 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 --2- 172.21.15.16:0/2377661191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e810be80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 -- 172.21.15.16:0/2377661191 >> 172.21.15.16:0/2377661191 conn(0x7fa5e807efd0 msgr2=0x7fa5e807f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 -- 172.21.15.16:0/2377661191 shutdown_connections 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.771+0000 7fa5ede00700 1 -- 172.21.15.16:0/2377661191 wait complete. 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5ede00700 1 Processor -- start 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5ede00700 1 -- start start 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5ede00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5ede00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810fff0 con 0x7fa5e810ba70 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5eca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:34.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.773+0000 7fa5eca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33826/0 (socket says 172.21.15.16:33826) 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.774+0000 7fa5eca00700 1 -- 172.21.15.16:0/2210545518 learned_addr learned my addr 172.21.15.16:0/2210545518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.774+0000 7fa5eca00700 1 -- 172.21.15.16:0/2210545518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e81a0330 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.774+0000 7fa5eca00700 1 --2- 172.21.15.16:0/2210545518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00046e0 tx=0x7fa5d0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d00039e0 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5d0003b80 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d0024700 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e81a05c0 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e81a31d0 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5d0022020 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.775+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5d0019490 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.776+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e8064e10 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.779+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa5d001ab30 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.857+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa5e81a34c0 con 0x7fa5e810ba70 2024-05-27T06:43:34.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.861+0000 7fa5e6000700 1 -- 172.21.15.16:0/2210545518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa5d001a430 con 0x7fa5e810ba70 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.862+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 msgr2=0x7fa5e819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.862+0000 7fa5ede00700 1 --2- 172.21.15.16:0/2210545518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00046e0 tx=0x7fa5d0011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.863+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 shutdown_connections 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.863+0000 7fa5ede00700 1 --2- 172.21.15.16:0/2210545518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e810ba70 0x7fa5e819fdf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.863+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 >> 172.21.15.16:0/2210545518 conn(0x7fa5e807efd0 msgr2=0x7fa5e8108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.863+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 shutdown_connections 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:33.863+0000 7fa5ede00700 1 -- 172.21.15.16:0/2210545518 wait complete. 2024-05-27T06:43:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T06:43:34.530 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T06:43:34.530 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-05-27T06:43:34.532 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T06:43:34.533 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T06:43:34.986 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T06:43:34.986 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-05-27T06:43:35.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:34 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2210545518' entity='client.admin' 2024-05-27T06:43:35.265 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968.target.wants/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@.service. 2024-05-27T06:43:35.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:35 smithi016 systemd[1]: Starting Ceph mgr.x for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:43:35.955 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:35 smithi016 podman[31630]: 2024-05-27 06:43:35.596089387 +0000 UTC m=+0.036318932 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T06:43:35.955 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:35 smithi016 podman[31630]: 2024-05-27 06:43:35.723923772 +0000 UTC m=+0.164153340 container create 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T06:43:35.955 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:35 smithi016 podman[31630]: 2024-05-27 06:43:35.960773216 +0000 UTC m=+0.401002847 container init 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T06:43:36.090 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T06:43:36.090 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-27T06:43:36.110 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T06:43:36.110 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:43:36.110 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-27T06:43:36.111 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T06:43:36.130 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T06:43:36.130 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-27T06:43:36.146 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T06:43:36.146 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:43:36.146 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-27T06:43:36.147 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-27T06:43:36.147 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-05-27T06:43:36.147 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-05-27T06:43:36.298 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:35 smithi016 podman[31630]: 2024-05-27 06:43:35.972003232 +0000 UTC m=+0.412232837 container start 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T06:43:36.298 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:36 smithi016 bash[31630]: 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c 2024-05-27T06:43:36.298 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:36 smithi016 systemd[1]: Started Ceph mgr.x for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:43:36.571 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:36.303+0000 7fd45a6c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T06:43:36.847 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:36.576+0000 7fd45a6c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:43:36.847 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:36.714+0000 7fd45a6c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T06:43:37.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:36 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3790278054' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T06:43:37.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:37.147+0000 7fd45a6c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T06:43:37.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:37.269+0000 7fd45a6c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T06:43:37.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T06:43:37.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T06:43:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T06:43:37.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T06:43:37.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T06:43:37.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T06:43:37.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:37.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T06:43:14.822394+0000", 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:43:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.852+0000 7fc6d8e00700 1 Processor -- start 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.852+0000 7fc6d8e00700 1 -- start start 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.853+0000 7fc6d8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.853+0000 7fc6d8e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d410a410 con 0x7fc6d4109a10 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.853+0000 7fc6d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.853+0000 7fc6d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33848/0 (socket says 172.21.15.16:33848) 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.853+0000 7fc6d1e00700 1 -- 172.21.15.16:0/3160403355 learned_addr learned my addr 172.21.15.16:0/3160403355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.854+0000 7fc6d1e00700 1 -- 172.21.15.16:0/3160403355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d410abe0 con 0x7fc6d4109a10 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.854+0000 7fc6d1e00700 1 --2- 172.21.15.16:0/3160403355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40098a0 tx=0x7fc6c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d880ab0fecbe1131 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.855+0000 7fc6d0a00700 1 -- 172.21.15.16:0/3160403355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4004160 con 0x7fc6d4109a10 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.855+0000 7fc6d0a00700 1 -- 172.21.15.16:0/3160403355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc6c4004300 con 0x7fc6d4109a10 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.855+0000 7fc6d0a00700 1 -- 172.21.15.16:0/3160403355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c40119e0 con 0x7fc6d4109a10 2024-05-27T06:43:37.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.855+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3160403355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 msgr2=0x7fc6d4109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.855+0000 7fc6d8e00700 1 --2- 172.21.15.16:0/3160403355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40098a0 tx=0x7fc6c4009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.856+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3160403355 shutdown_connections 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.856+0000 7fc6d8e00700 1 --2- 172.21.15.16:0/3160403355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d4109e20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.856+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3160403355 >> 172.21.15.16:0/3160403355 conn(0x7fc6d4105050 msgr2=0x7fc6d4107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.856+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3160403355 shutdown_connections 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.856+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3160403355 wait complete. 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.857+0000 7fc6d8e00700 1 Processor -- start 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d8e00700 1 -- start start 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d8e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d410df90 con 0x7fc6d4109a10 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:37.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33856/0 (socket says 172.21.15.16:33856) 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.858+0000 7fc6d1e00700 1 -- 172.21.15.16:0/3790278054 learned_addr learned my addr 172.21.15.16:0/3790278054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.859+0000 7fc6d1e00700 1 -- 172.21.15.16:0/3790278054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d407f9e0 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.859+0000 7fc6d1e00700 1 --2- 172.21.15.16:0/3790278054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40046e0 tx=0x7fc6c4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.859+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c40039e0 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.859+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc6c4003b80 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.859+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d407dc30 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.860+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d407e110 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.860+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4024700 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.860+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc6c4022020 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.860+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc6c402dd00 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.861+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b4005360 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.864+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc6c401ab30 con 0x7fc6d4109a10 2024-05-27T06:43:37.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.944+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc6b4005920 con 0x7fc6d4109a10 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.945+0000 7fc6caa00700 1 -- 172.21.15.16:0/3790278054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc6c402d3f0 con 0x7fc6d4109a10 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 msgr2=0x7fc6d407f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 --2- 172.21.15.16:0/3790278054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40046e0 tx=0x7fc6c4011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 shutdown_connections 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 --2- 172.21.15.16:0/3790278054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d4109a10 0x7fc6d407f4a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 >> 172.21.15.16:0/3790278054 conn(0x7fc6d4105050 msgr2=0x7fc6d4105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.946+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 shutdown_connections 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:36.947+0000 7fc6d8e00700 1 -- 172.21.15.16:0/3790278054 wait complete. 2024-05-27T06:43:37.674 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-05-27T06:43:39.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:38 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:38.936+0000 7fd45a6c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T06:43:39.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:39.173+0000 7fd45a6c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T06:43:39.803 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:39.393+0000 7fd45a6c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T06:43:39.803 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:39.808+0000 7fd45a6c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T06:43:40.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:39.938+0000 7fd45a6c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T06:43:40.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:40.212+0000 7fd45a6c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T06:43:40.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:40 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/827207527' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T06:43:40.842 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:40.490+0000 7fd45a6c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T06:43:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T06:43:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T06:43:41.292 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T06:43:41.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T06:43:14.822394+0000", 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T06:43:41.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.324+0000 7f33b0e00700 1 Processor -- start 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.324+0000 7f33b0e00700 1 -- start start 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.324+0000 7f33b0e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.324+0000 7f33b0e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac10a450 con 0x7f33ac109a50 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.325+0000 7f33a9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.325+0000 7f33a9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33858/0 (socket says 172.21.15.16:33858) 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.325+0000 7f33a9e00700 1 -- 172.21.15.16:0/2717507335 learned_addr learned my addr 172.21.15.16:0/2717507335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.325+0000 7f33a9e00700 1 -- 172.21.15.16:0/2717507335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac10ac20 con 0x7f33ac109a50 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.326+0000 7f33a9e00700 1 --2- 172.21.15.16:0/2717507335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f339c0098a0 tx=0x7f339c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fccda5044ef1183 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.326+0000 7f33a8a00700 1 -- 172.21.15.16:0/2717507335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c004160 con 0x7f33ac109a50 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.326+0000 7f33a8a00700 1 -- 172.21.15.16:0/2717507335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f339c004300 con 0x7f33ac109a50 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.326+0000 7f33a8a00700 1 -- 172.21.15.16:0/2717507335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c0119e0 con 0x7f33ac109a50 2024-05-27T06:43:41.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 -- 172.21.15.16:0/2717507335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 msgr2=0x7f33ac109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 --2- 172.21.15.16:0/2717507335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f339c0098a0 tx=0x7f339c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 -- 172.21.15.16:0/2717507335 shutdown_connections 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 --2- 172.21.15.16:0/2717507335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac109e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 -- 172.21.15.16:0/2717507335 >> 172.21.15.16:0/2717507335 conn(0x7f33ac1050b0 msgr2=0x7f33ac107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 -- 172.21.15.16:0/2717507335 shutdown_connections 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.327+0000 7f33b0e00700 1 -- 172.21.15.16:0/2717507335 wait complete. 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.329+0000 7f33b0e00700 1 Processor -- start 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.329+0000 7f33b0e00700 1 -- start start 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.329+0000 7f33b0e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.329+0000 7f33b0e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac10dfd0 con 0x7f33ac109a50 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.330+0000 7f33a9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.330+0000 7f33a9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33866/0 (socket says 172.21.15.16:33866) 2024-05-27T06:43:41.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.330+0000 7f33a9e00700 1 -- 172.21.15.16:0/827207527 learned_addr learned my addr 172.21.15.16:0/827207527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.330+0000 7f33a9e00700 1 -- 172.21.15.16:0/827207527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac07efb0 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.330+0000 7f33a9e00700 1 --2- 172.21.15.16:0/827207527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f339c009fb0 tx=0x7f339c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c0038b0 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f339c003a50 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c01a5c0 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac07d200 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac07d6e0 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f339c01ac00 con 0x7f33ac109a50 2024-05-27T06:43:41.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.331+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f339c019700 con 0x7f33ac109a50 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.332+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac064e10 con 0x7f33ac109a50 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.335+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f339c024a60 con 0x7f33ac109a50 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.417+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f33ac07de30 con 0x7f33ac109a50 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.418+0000 7f33a2a00700 1 -- 172.21.15.16:0/827207527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f339c024440 con 0x7f33ac109a50 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 msgr2=0x7f33ac07ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 --2- 172.21.15.16:0/827207527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f339c009fb0 tx=0x7f339c011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 shutdown_connections 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 --2- 172.21.15.16:0/827207527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ac109a50 0x7f33ac07ea70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 >> 172.21.15.16:0/827207527 conn(0x7f33ac1050b0 msgr2=0x7f33ac105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.419+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 shutdown_connections 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:40.420+0000 7f33b0e00700 1 -- 172.21.15.16:0/827207527 wait complete. 2024-05-27T06:43:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-05-27T06:43:41.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:41.329+0000 7fd45a6c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T06:43:42.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:41.628+0000 7fd45a6c1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T06:43:42.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:42.202+0000 7fd45a6c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T06:43:42.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:42.334+0000 7fd45a6c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T06:43:43.077 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:42.817+0000 7fd45a6c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T06:43:43.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:43.293+0000 7fd45a6c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T06:43:43.734 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:43.423+0000 7fd45a6c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T06:43:44.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:43.766+0000 7fd45a6c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T06:43:44.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:43.959+0000 7fd45a6c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T06:43:44.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/921501189' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T06:43:44.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:44.097+0000 7fd45a6c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T06:43:44.641 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:44.516+0000 7fd45a6c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T06:43:44.641 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:44.647+0000 7fd45a6c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T06:43:44.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:44.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:43:44.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T06:43:44.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T06:43:44.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T06:43:44.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T06:43:44.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T06:43:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:44.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T06:43:14.822394+0000", 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.970+0000 7f1ecca00700 1 Processor -- start 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.970+0000 7f1ecca00700 1 -- start start 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ecca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ecca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810a450 con 0x7f1ec8109a50 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ec5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:44.692 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ec5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53434/0 (socket says 172.21.15.16:53434) 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ec5e00700 1 -- 172.21.15.16:0/3981808971 learned_addr learned my addr 172.21.15.16:0/3981808971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.971+0000 7f1ec5e00700 1 -- 172.21.15.16:0/3981808971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec810ac20 con 0x7f1ec8109a50 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.972+0000 7f1ec5e00700 1 --2- 172.21.15.16:0/3981808971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80098a0 tx=0x7f1eb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82f67d77fedd3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.972+0000 7f1ec4a00700 1 -- 172.21.15.16:0/3981808971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb8004160 con 0x7f1ec8109a50 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.972+0000 7f1ec4a00700 1 -- 172.21.15.16:0/3981808971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1eb8004300 con 0x7f1ec8109a50 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.972+0000 7f1ec4a00700 1 -- 172.21.15.16:0/3981808971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb80119e0 con 0x7f1ec8109a50 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 -- 172.21.15.16:0/3981808971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 msgr2=0x7f1ec8109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 --2- 172.21.15.16:0/3981808971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80098a0 tx=0x7f1eb8009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 -- 172.21.15.16:0/3981808971 shutdown_connections 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 --2- 172.21.15.16:0/3981808971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec8109e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 -- 172.21.15.16:0/3981808971 >> 172.21.15.16:0/3981808971 conn(0x7f1ec81050b0 msgr2=0x7f1ec8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 -- 172.21.15.16:0/3981808971 shutdown_connections 2024-05-27T06:43:44.693 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.973+0000 7f1ecca00700 1 -- 172.21.15.16:0/3981808971 wait complete. 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.975+0000 7f1ecca00700 1 Processor -- start 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.975+0000 7f1ecca00700 1 -- start start 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.975+0000 7f1ecca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ecca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810dfd0 con 0x7f1ec8109a50 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ec5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ec5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53440/0 (socket says 172.21.15.16:53440) 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ec5e00700 1 -- 172.21.15.16:0/921501189 learned_addr learned my addr 172.21.15.16:0/921501189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ec5e00700 1 -- 172.21.15.16:0/921501189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec807efb0 con 0x7f1ec8109a50 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.976+0000 7f1ec5e00700 1 --2- 172.21.15.16:0/921501189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80046e0 tx=0x7f1eb8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb80039e0 con 0x7f1ec8109a50 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1eb8003b80 con 0x7f1ec8109a50 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec807d200 con 0x7f1ec8109a50 2024-05-27T06:43:44.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb8024700 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec807d6e0 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1eb8022020 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.977+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1eb802dce0 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.978+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8064e10 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:43.981+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1eb801ab30 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.062+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1ec807de30 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.062+0000 7f1ebea00700 1 -- 172.21.15.16:0/921501189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1eb802d3f0 con 0x7f1ec8109a50 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 msgr2=0x7f1ec807ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 --2- 172.21.15.16:0/921501189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80046e0 tx=0x7f1eb8011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 shutdown_connections 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 --2- 172.21.15.16:0/921501189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8109a50 0x7f1ec807ea70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 >> 172.21.15.16:0/921501189 conn(0x7f1ec81050b0 msgr2=0x7f1ec8105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 shutdown_connections 2024-05-27T06:43:44.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:44.064+0000 7f1ecca00700 1 -- 172.21.15.16:0/921501189 wait complete. 2024-05-27T06:43:44.696 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (3/15)... 2024-05-27T06:43:45.744 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:45.450+0000 7fd45a6c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T06:43:46.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:45.749+0000 7fd45a6c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T06:43:46.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: Activating manager daemon x 2024-05-27T06:43:46.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: mgrmap e2: x(active, starting, since 0.0133069s) 2024-05-27T06:43:46.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: Manager daemon x is now available 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T06:43:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:45 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' 2024-05-27T06:43:47.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:46 smithi016 ceph-mon[31415]: from='mgr.14100 172.21.15.16:0/3154076482' entity='mgr.x' 2024-05-27T06:43:47.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:46 smithi016 ceph-mon[31415]: mgrmap e3: x(active, since 1.01863s) 2024-05-27T06:43:48.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:47 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3920526369' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T06:43:48.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:48.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:43:48.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:43:48.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T06:43:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T06:43:48.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T06:43:48.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T06:43:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T06:43:14.822394+0000", 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.430+0000 7fa4f3c00700 1 Processor -- start 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.431+0000 7fa4f3c00700 1 -- start start 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.431+0000 7fa4f3c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.431+0000 7fa4f3c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec10c600 con 0x7fa4ec10bc00 2024-05-27T06:43:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.431+0000 7fa4f1200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.432+0000 7fa4f1200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53508/0 (socket says 172.21.15.16:53508) 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.432+0000 7fa4f1200700 1 -- 172.21.15.16:0/1186219079 learned_addr learned my addr 172.21.15.16:0/1186219079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.432+0000 7fa4f1200700 1 -- 172.21.15.16:0/1186219079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec10cdd0 con 0x7fa4ec10bc00 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.432+0000 7fa4f1200700 1 --2- 172.21.15.16:0/1186219079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00098a0 tx=0x7fa4e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c0b6fafb5ac35a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.433+0000 7fa4eb400700 1 -- 172.21.15.16:0/1186219079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e0004160 con 0x7fa4ec10bc00 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.433+0000 7fa4eb400700 1 -- 172.21.15.16:0/1186219079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa4e0004300 con 0x7fa4ec10bc00 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.433+0000 7fa4eb400700 1 -- 172.21.15.16:0/1186219079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e00119e0 con 0x7fa4ec10bc00 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 -- 172.21.15.16:0/1186219079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 msgr2=0x7fa4ec10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/1186219079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00098a0 tx=0x7fa4e0009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 -- 172.21.15.16:0/1186219079 shutdown_connections 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/1186219079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec10c010 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 -- 172.21.15.16:0/1186219079 >> 172.21.15.16:0/1186219079 conn(0x7fa4ec1074c0 msgr2=0x7fa4ec1098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 -- 172.21.15.16:0/1186219079 shutdown_connections 2024-05-27T06:43:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.434+0000 7fa4f3c00700 1 -- 172.21.15.16:0/1186219079 wait complete. 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.435+0000 7fa4f3c00700 1 Processor -- start 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.435+0000 7fa4f3c00700 1 -- start start 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f3c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f3c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec110180 con 0x7fa4ec10bc00 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f1200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f1200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53522/0 (socket says 172.21.15.16:53522) 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f1200700 1 -- 172.21.15.16:0/3920526369 learned_addr learned my addr 172.21.15.16:0/3920526369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.436+0000 7fa4f1200700 1 -- 172.21.15.16:0/3920526369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec1a04c0 con 0x7fa4ec10bc00 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4f1200700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00046e0 tx=0x7fa4e0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e00039e0 con 0x7fa4ec10bc00 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa4e0003b80 con 0x7fa4ec10bc00 2024-05-27T06:43:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec1a0750 con 0x7fa4ec10bc00 2024-05-27T06:43:48.446 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e0024700 con 0x7fa4ec10bc00 2024-05-27T06:43:48.446 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.437+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a3360 con 0x7fa4ec10bc00 2024-05-27T06:43:48.446 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.439+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fa4e0022020 con 0x7fa4ec10bc00 2024-05-27T06:43:48.446 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.439+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec064e10 con 0x7fa4ec10bc00 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.439+0000 7fa4e9600700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 0x7fa4d4040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.439+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4e0053340 con 0x7fa4ec10bc00 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.440+0000 7fa4ebe00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 0x7fa4d4040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.441+0000 7fa4ebe00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 0x7fa4d4040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8006fd0 tx=0x7fa4d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.446+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4e000b3e0 con 0x7fa4ec10bc00 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.774+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa4ec1a3580 con 0x7fa4ec10bc00 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.776+0000 7fa4e9600700 1 -- 172.21.15.16:0/3920526369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa4ec1a3580 con 0x7fa4ec10bc00 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.778+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 msgr2=0x7fa4d4040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.778+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 0x7fa4d4040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8006fd0 tx=0x7fa4d8006e40 comp rx=0 tx=0).stop 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.778+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 msgr2=0x7fa4ec19ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.778+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00046e0 tx=0x7fa4e0011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 shutdown_connections 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fa4d403e350 0x7fa4d4040860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:48.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 --2- 172.21.15.16:0/3920526369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4ec10bc00 0x7fa4ec19ff80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:48.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 >> 172.21.15.16:0/3920526369 conn(0x7fa4ec1074c0 msgr2=0x7fa4ec108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:48.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 shutdown_connections 2024-05-27T06:43:48.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:47.779+0000 7fa4f3c00700 1 -- 172.21.15.16:0/3920526369 wait complete. 2024-05-27T06:43:48.449 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-05-27T06:43:49.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:48 smithi016 ceph-mon[31415]: mgrmap e4: x(active, since 2s) 2024-05-27T06:43:50.098 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:49 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1350052324' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T06:43:50.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:50.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T06:43:50.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:43:50.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T06:43:50.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T06:43:50.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.153+0000 7f7827400700 1 Processor -- start 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.153+0000 7f7827400700 1 -- start start 2024-05-27T06:43:50.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.153+0000 7f7827400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:50.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.153+0000 7f7827400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f782010a450 con 0x7f7820109a50 2024-05-27T06:43:50.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.154+0000 7f7824a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:50.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.154+0000 7f7824a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53532/0 (socket says 172.21.15.16:53532) 2024-05-27T06:43:50.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.154+0000 7f7824a00700 1 -- 172.21.15.16:0/4138304705 learned_addr learned my addr 172.21.15.16:0/4138304705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.154+0000 7f7824a00700 1 -- 172.21.15.16:0/4138304705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782010ac20 con 0x7f7820109a50 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.155+0000 7f7824a00700 1 --2- 172.21.15.16:0/4138304705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f7810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3a377b913c55c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.155+0000 7f781fe00700 1 -- 172.21.15.16:0/4138304705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810004160 con 0x7f7820109a50 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.155+0000 7f781fe00700 1 -- 172.21.15.16:0/4138304705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7810004300 con 0x7f7820109a50 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.155+0000 7f781fe00700 1 -- 172.21.15.16:0/4138304705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78100119e0 con 0x7f7820109a50 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 -- 172.21.15.16:0/4138304705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 msgr2=0x7f7820109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 --2- 172.21.15.16:0/4138304705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f7810009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 -- 172.21.15.16:0/4138304705 shutdown_connections 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 --2- 172.21.15.16:0/4138304705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f7820109e60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 -- 172.21.15.16:0/4138304705 >> 172.21.15.16:0/4138304705 conn(0x7f78201050b0 msgr2=0x7f7820107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 -- 172.21.15.16:0/4138304705 shutdown_connections 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.156+0000 7f7827400700 1 -- 172.21.15.16:0/4138304705 wait complete. 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.157+0000 7f7827400700 1 Processor -- start 2024-05-27T06:43:50.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7827400700 1 -- start start 2024-05-27T06:43:50.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7827400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:50.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7827400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f782010dfd0 con 0x7f7820109a50 2024-05-27T06:43:50.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7824a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7824a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53536/0 (socket says 172.21.15.16:53536) 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.158+0000 7f7824a00700 1 -- 172.21.15.16:0/1350052324 learned_addr learned my addr 172.21.15.16:0/1350052324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f7824a00700 1 -- 172.21.15.16:0/1350052324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782019e330 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f7824a00700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7810009fb0 tx=0x7f7810011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78100038b0 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7810003a50 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f781001a5c0 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.159+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782019e5c0 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.160+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201a1190 con 0x7f7820109a50 2024-05-27T06:43:50.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.161+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820064e10 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.161+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f7810024440 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.161+0000 7f781e000700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 0x7f7804040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.161+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7810053a30 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.162+0000 7f7817e00700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 0x7f7804040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.162+0000 7f7817e00700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 0x7f7804040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.167+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78100212a0 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.404+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f78201a2a30 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.406+0000 7f781e000700 1 -- 172.21.15.16:0/1350052324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f7810053450 con 0x7f7820109a50 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 msgr2=0x7f7804040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 0x7f7804040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).stop 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 msgr2=0x7f782019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7810009fb0 tx=0x7f7810011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 shutdown_connections 2024-05-27T06:43:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f780403e420 0x7f7804040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:50.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 --2- 172.21.15.16:0/1350052324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820109a50 0x7f782019ddf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:50.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.409+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 >> 172.21.15.16:0/1350052324 conn(0x7f78201050b0 msgr2=0x7f7820105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:50.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.410+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 shutdown_connections 2024-05-27T06:43:50.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:49.410+0000 7f7827400700 1 -- 172.21.15.16:0/1350052324 wait complete. 2024-05-27T06:43:50.122 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-05-27T06:43:51.401 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:51 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3231563711' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T06:43:51.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setuser ceph since I am not root 2024-05-27T06:43:51.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setgroup ceph since I am not root 2024-05-27T06:43:51.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:51.582+0000 7f8825aab200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T06:43:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.830+0000 7f0b73000700 1 Processor -- start 2024-05-27T06:43:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.831+0000 7f0b73000700 1 -- start start 2024-05-27T06:43:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.831+0000 7f0b73000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.831+0000 7f0b73000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c113090 con 0x7f0b6c110730 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.832+0000 7f0b71c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.832+0000 7f0b71c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53544/0 (socket says 172.21.15.16:53544) 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.832+0000 7f0b71c00700 1 -- 172.21.15.16:0/239034612 learned_addr learned my addr 172.21.15.16:0/239034612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.832+0000 7f0b71c00700 1 -- 172.21.15.16:0/239034612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c113210 con 0x7f0b6c110730 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.833+0000 7f0b71c00700 1 --2- 172.21.15.16:0/239034612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c197bc449cc1fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.833+0000 7f0b6be00700 1 -- 172.21.15.16:0/239034612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b60004160 con 0x7f0b6c110730 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.833+0000 7f0b6be00700 1 -- 172.21.15.16:0/239034612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0b60004300 con 0x7f0b6c110730 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.833+0000 7f0b6be00700 1 -- 172.21.15.16:0/239034612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b600119e0 con 0x7f0b6c110730 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 -- 172.21.15.16:0/239034612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 msgr2=0x7f0b6c112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 --2- 172.21.15.16:0/239034612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 -- 172.21.15.16:0/239034612 shutdown_connections 2024-05-27T06:43:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 --2- 172.21.15.16:0/239034612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c112b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 -- 172.21.15.16:0/239034612 >> 172.21.15.16:0/239034612 conn(0x7f0b6c07efd0 msgr2=0x7f0b6c0813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 -- 172.21.15.16:0/239034612 shutdown_connections 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.834+0000 7f0b73000700 1 -- 172.21.15.16:0/239034612 wait complete. 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.835+0000 7f0b73000700 1 Processor -- start 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.835+0000 7f0b73000700 1 -- start start 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b73000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b73000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c1141e0 con 0x7f0b6c110730 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b71c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b71c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53556/0 (socket says 172.21.15.16:53556) 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b71c00700 1 -- 172.21.15.16:0/3231563711 learned_addr learned my addr 172.21.15.16:0/3231563711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.836+0000 7f0b71c00700 1 -- 172.21.15.16:0/3231563711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c1a46c0 con 0x7f0b6c110730 2024-05-27T06:43:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b71c00700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b600116a0 tx=0x7f0b60011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b60003970 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0b60003b10 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b60019550 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c1a4950 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.837+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c1a7560 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.839+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f0b6001aa50 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.839+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c064e10 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.839+0000 7f0b6a000700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 0x7f0b54040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.839+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b600537e0 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.840+0000 7f0b71200700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 0x7f0b54040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.840+0000 7f0b71200700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 0x7f0b54040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b58006fd0 tx=0x7f0b58006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:50.845+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b60053200 con 0x7f0b6c110730 2024-05-27T06:43:51.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.143+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f0b6c1a7780 con 0x7f0b6c110730 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.196+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0b6001a430 con 0x7f0b6c110730 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.200+0000 7f0b6a000700 1 -- 172.21.15.16:0/3231563711 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f0b60018a40 con 0x7f0b6c110730 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.200+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 msgr2=0x7f0b54040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.200+0000 7f0b73000700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 0x7f0b54040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b58006fd0 tx=0x7f0b58006e40 comp rx=0 tx=0).stop 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.200+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 msgr2=0x7f0b6c1a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.200+0000 7f0b73000700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b600116a0 tx=0x7f0b60011a60 comp rx=0 tx=0).stop 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.201+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 shutdown_connections 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.201+0000 7f0b73000700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f0b5403e470 0x7f0b54040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.201+0000 7f0b73000700 1 --2- 172.21.15.16:0/3231563711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c110730 0x7f0b6c1a4180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.201+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 >> 172.21.15.16:0/3231563711 conn(0x7f0b6c07efd0 msgr2=0x7f0b6c080890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.201+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 shutdown_connections 2024-05-27T06:43:51.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:51.202+0000 7f0b73000700 1 -- 172.21.15.16:0/3231563711 wait complete. 2024-05-27T06:43:52.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:51.830+0000 7f8825aab200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:43:52.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:51.969+0000 7f8825aab200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T06:43:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:52 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3231563711' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T06:43:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:52 smithi016 ceph-mon[31415]: mgrmap e5: x(active, since 5s) 2024-05-27T06:43:52.697 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:52.383+0000 7f8825aab200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T06:43:52.697 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:52.503+0000 7f8825aab200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T06:43:53.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:43:53 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/948150925' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.703+0000 7fd889400700 1 Processor -- start 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.703+0000 7fd889400700 1 -- start start 2024-05-27T06:43:53.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.704+0000 7fd889400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.704+0000 7fd889400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd88407d850 con 0x7fd88407ee50 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.704+0000 7fd883e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.704+0000 7fd883e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59416/0 (socket says 172.21.15.16:59416) 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.704+0000 7fd883e00700 1 -- 172.21.15.16:0/3294051115 learned_addr learned my addr 172.21.15.16:0/3294051115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.705+0000 7fd883e00700 1 -- 172.21.15.16:0/3294051115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88407d9d0 con 0x7fd88407ee50 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.705+0000 7fd883e00700 1 --2- 172.21.15.16:0/3294051115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d4833826955331 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.705+0000 7fd882a00700 1 -- 172.21.15.16:0/3294051115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874004160 con 0x7fd88407ee50 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.705+0000 7fd882a00700 1 -- 172.21.15.16:0/3294051115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd874004300 con 0x7fd88407ee50 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.705+0000 7fd882a00700 1 -- 172.21.15.16:0/3294051115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8740119e0 con 0x7fd88407ee50 2024-05-27T06:43:53.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.706+0000 7fd889400700 1 -- 172.21.15.16:0/3294051115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 msgr2=0x7fd88407d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.706+0000 7fd889400700 1 --2- 172.21.15.16:0/3294051115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).stop 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.706+0000 7fd889400700 1 -- 172.21.15.16:0/3294051115 shutdown_connections 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.706+0000 7fd889400700 1 --2- 172.21.15.16:0/3294051115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88407d280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.706+0000 7fd889400700 1 -- 172.21.15.16:0/3294051115 >> 172.21.15.16:0/3294051115 conn(0x7fd884104f90 msgr2=0x7fd8841073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.707+0000 7fd889400700 1 -- 172.21.15.16:0/3294051115 shutdown_connections 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.707+0000 7fd889400700 1 -- 172.21.15.16:0/3294051115 wait complete. 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.708+0000 7fd889400700 1 Processor -- start 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.708+0000 7fd889400700 1 -- start start 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.708+0000 7fd889400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.708+0000 7fd889400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd88410fe30 con 0x7fd88407ee50 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.708+0000 7fd883e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.709+0000 7fd883e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59426/0 (socket says 172.21.15.16:59426) 2024-05-27T06:43:53.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.709+0000 7fd883e00700 1 -- 172.21.15.16:0/948150925 learned_addr learned my addr 172.21.15.16:0/948150925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.709+0000 7fd883e00700 1 -- 172.21.15.16:0/948150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8841a0330 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.709+0000 7fd883e00700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd8740119b0 tx=0x7fd874011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.709+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874003af0 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.710+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd87401a8a0 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.710+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8841a05c0 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.710+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd874024830 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.710+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8841a31d0 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.711+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd884064e10 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.711+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd874022020 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.712+0000 7fd880c00700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 0x7fd868040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.712+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd874053610 con 0x7fd88407ee50 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.712+0000 7fd883400700 1 -- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 msgr2=0x7fd868040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:43:53.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.712+0000 7fd883400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 0x7fd868040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.718+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8740532f0 con 0x7fd88407ee50 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.913+0000 7fd883400700 1 -- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 msgr2=0x7fd868040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:52.913+0000 7fd883400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 0x7fd868040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.014+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd8841a34c0 con 0x7fd88407ee50 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.015+0000 7fd880c00700 1 -- 172.21.15.16:0/948150925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd87401aa50 con 0x7fd88407ee50 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 msgr2=0x7fd868040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 0x7fd868040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 msgr2=0x7fd88419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd8740119b0 tx=0x7fd874011a90 comp rx=0 tx=0).stop 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 shutdown_connections 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7fd86803e470 0x7fd868040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 --2- 172.21.15.16:0/948150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88407ee50 0x7fd88419fdf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:43:53.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.018+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 >> 172.21.15.16:0/948150925 conn(0x7fd884104f90 msgr2=0x7fd8841068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:43:53.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.019+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 shutdown_connections 2024-05-27T06:43:53.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:53.019+0000 7fd889400700 1 -- 172.21.15.16:0/948150925 wait complete. 2024-05-27T06:43:53.838 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-05-27T06:43:53.838 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-05-27T06:43:54.391 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:54.157+0000 7f8825aab200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T06:43:54.391 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:54.396+0000 7f8825aab200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T06:43:54.749 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:54.621+0000 7f8825aab200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T06:43:55.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:55.021+0000 7f8825aab200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T06:43:55.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:55.149+0000 7f8825aab200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T06:43:55.671 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:55.410+0000 7f8825aab200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T06:43:55.951 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:55.676+0000 7f8825aab200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:43:56.794 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:56.500+0000 7f8825aab200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T06:43:57.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:56.800+0000 7f8825aab200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T06:43:57.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:57.378+0000 7f8825aab200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T06:43:57.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:57.511+0000 7f8825aab200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T06:43:58.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:57.988+0000 7f8825aab200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T06:43:58.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:58.459+0000 7f8825aab200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T06:43:58.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:58.590+0000 7f8825aab200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T06:43:59.243 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:58.926+0000 7f8825aab200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T06:43:59.243 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:59.114+0000 7f8825aab200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T06:43:59.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:59.248+0000 7f8825aab200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T06:44:00.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:59.657+0000 7f8825aab200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T06:44:00.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:43:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:43:59.785+0000 7f8825aab200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T06:44:00.756 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:00.585+0000 7f8825aab200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T06:44:01.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:00.889+0000 7f8825aab200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: Active manager daemon x restarted 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: Activating manager daemon x 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: mgrmap e6: x(active, starting, since 0.00596962s) 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T06:44:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T06:44:01.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: Manager daemon x is now available 2024-05-27T06:44:01.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:00 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:02.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: Found migration_current of "None". Setting to last migration. 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:01 smithi016 ceph-mon[31415]: mgrmap e7: x(active, since 1.01993s) 2024-05-27T06:44:02.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.544+0000 7f74bfa00700 1 Processor -- start 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.544+0000 7f74bfa00700 1 -- start start 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.545+0000 7f74bfa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.545+0000 7f74bfa00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b807d870 con 0x7f74b807ee70 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.545+0000 7f74bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.545+0000 7f74bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59440/0 (socket says 172.21.15.16:59440) 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.545+0000 7f74bd000700 1 -- 172.21.15.16:0/3614463324 learned_addr learned my addr 172.21.15.16:0/3614463324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.546+0000 7f74bd000700 1 -- 172.21.15.16:0/3614463324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b807d9f0 con 0x7f74b807ee70 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.546+0000 7f74bd000700 1 --2- 172.21.15.16:0/3614463324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098a0 tx=0x7f74ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aeb872cae5ee47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.547+0000 7f74b7400700 1 -- 172.21.15.16:0/3614463324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ac004160 con 0x7f74b807ee70 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.547+0000 7f74b7400700 1 -- 172.21.15.16:0/3614463324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f74ac004300 con 0x7f74b807ee70 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.547+0000 7f74b7400700 1 -- 172.21.15.16:0/3614463324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ac0119e0 con 0x7f74b807ee70 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.547+0000 7f74bfa00700 1 -- 172.21.15.16:0/3614463324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 msgr2=0x7f74b807d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.547+0000 7f74bfa00700 1 --2- 172.21.15.16:0/3614463324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098a0 tx=0x7f74ac009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.548+0000 7f74bfa00700 1 -- 172.21.15.16:0/3614463324 shutdown_connections 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.548+0000 7f74bfa00700 1 --2- 172.21.15.16:0/3614463324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b807d2a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.548+0000 7f74bfa00700 1 -- 172.21.15.16:0/3614463324 >> 172.21.15.16:0/3614463324 conn(0x7f74b81050c0 msgr2=0x7f74b8107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.548+0000 7f74bfa00700 1 -- 172.21.15.16:0/3614463324 shutdown_connections 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.548+0000 7f74bfa00700 1 -- 172.21.15.16:0/3614463324 wait complete. 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.549+0000 7f74bfa00700 1 Processor -- start 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.550+0000 7f74bfa00700 1 -- start start 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.550+0000 7f74bfa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.550+0000 7f74bfa00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b810bc40 con 0x7f74b807ee70 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.550+0000 7f74bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.551+0000 7f74bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59446/0 (socket says 172.21.15.16:59446) 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.551+0000 7f74bd000700 1 -- 172.21.15.16:0/866435834 learned_addr learned my addr 172.21.15.16:0/866435834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:02.650 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.551+0000 7f74bd000700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b81a4940 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.551+0000 7f74bd000700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0119b0 tx=0x7f74ac011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.552+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ac003af0 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.552+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f74ac01a8a0 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.552+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b81a4bd0 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.552+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ac024830 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.552+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b81a77e0 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f74ac022020 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b5600700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f74a00410e0 con 0x7f74a003e4c0 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f74ac053540 con 0x7f74b807ee70 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.554+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T06:44:02.651 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.755+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.652 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:54.755+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T06:44:02.652 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:55.156+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:55.156+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:55.957+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:55.957+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:57.559+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:43:57.559+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:00.763+0000 7f74b7e00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4120962675 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:00.763+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:00.898+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f74ac01aa50 con 0x7f74b807ee70 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:00.898+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 msgr2=0x7f74a00409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:00.898+0000 7f74b5600700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.906+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f74ac055010 con 0x7f74b807ee70 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.906+0000 7f74b5600700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 0x7f74a0044220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:02.655 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.906+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f74a00410e0 con 0x7f74a0041e00 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.908+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 0x7f74a0044220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.911+0000 7f74b7e00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 0x7f74a0044220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f74a4003a10 tx=0x7f74a40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.913+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f74a00410e0 con 0x7f74a0041e00 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.929+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f74b8064e10 con 0x7f74a0041e00 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.929+0000 7f74b5600700 1 -- 172.21.15.16:0/866435834 <== mgr.14120 v2:172.21.15.16:6800/2390178341 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f74b8064e10 con 0x7f74a0041e00 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 msgr2=0x7f74a0044220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 0x7f74a0044220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f74a4003a10 tx=0x7f74a40092b0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 msgr2=0x7f74b81a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0119b0 tx=0x7f74ac011a90 comp rx=0 tx=0).stop 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 shutdown_connections 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f74a0041e00 0x7f74a0044220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b807ee70 0x7f74b81a4400 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 --2- 172.21.15.16:0/866435834 >> [v2:172.21.15.16:6800/4120962675,v1:172.21.15.16:6801/4120962675] conn(0x7f74a003e4c0 0x7f74a00409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 >> 172.21.15.16:0/866435834 conn(0x7f74b81050c0 msgr2=0x7f74b819ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:02.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.930+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 shutdown_connections 2024-05-27T06:44:02.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:01.931+0000 7f74bfa00700 1 -- 172.21.15.16:0/866435834 wait complete. 2024-05-27T06:44:02.657 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-05-27T06:44:02.657 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-05-27T06:44:03.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:02 smithi016 ceph-mon[31415]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T06:44:03.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:02 smithi016 ceph-mon[31415]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T06:44:03.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:02 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: [27/May/2024:06:44:02] ENGINE Bus STARTING 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: [27/May/2024:06:44:02] ENGINE Serving on http://172.21.15.16:8765 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: [27/May/2024:06:44:02] ENGINE Serving on https://172.21.15.16:7150 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: [27/May/2024:06:44:02] ENGINE Bus STARTED 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: mgrmap e8: x(active, since 2s) 2024-05-27T06:44:04.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:04.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:03 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.378+0000 7f017b400700 1 Processor -- start 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.378+0000 7f017b400700 1 -- start start 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.378+0000 7f017b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.378+0000 7f017b400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0174113200 con 0x7f01741107f0 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.379+0000 7f0178a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.379+0000 7f0178a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59186/0 (socket says 172.21.15.16:59186) 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.379+0000 7f0178a00700 1 -- 172.21.15.16:0/1623517720 learned_addr learned my addr 172.21.15.16:0/1623517720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.379+0000 7f0178a00700 1 -- 172.21.15.16:0/1623517720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0174113380 con 0x7f01741107f0 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.380+0000 7f0178a00700 1 --2- 172.21.15.16:0/1623517720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01680098a0 tx=0x7f0168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0db0364920e1abd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.380+0000 7f0173400700 1 -- 172.21.15.16:0/1623517720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0168004160 con 0x7f01741107f0 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.380+0000 7f0173400700 1 -- 172.21.15.16:0/1623517720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0168004300 con 0x7f01741107f0 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.380+0000 7f0173400700 1 -- 172.21.15.16:0/1623517720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01680119e0 con 0x7f01741107f0 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 -- 172.21.15.16:0/1623517720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 msgr2=0x7f0174112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 --2- 172.21.15.16:0/1623517720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01680098a0 tx=0x7f0168009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 -- 172.21.15.16:0/1623517720 shutdown_connections 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 --2- 172.21.15.16:0/1623517720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f0174112c10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 -- 172.21.15.16:0/1623517720 >> 172.21.15.16:0/1623517720 conn(0x7f017407f0b0 msgr2=0x7f0174081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 -- 172.21.15.16:0/1623517720 shutdown_connections 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.381+0000 7f017b400700 1 -- 172.21.15.16:0/1623517720 wait complete. 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.382+0000 7f017b400700 1 Processor -- start 2024-05-27T06:44:04.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.382+0000 7f017b400700 1 -- start start 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f017b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f017b400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01741143a0 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f0178a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f0178a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59188/0 (socket says 172.21.15.16:59188) 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f0178a00700 1 -- 172.21.15.16:0/2859312707 learned_addr learned my addr 172.21.15.16:0/2859312707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.383+0000 7f0178a00700 1 -- 172.21.15.16:0/2859312707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741a48a0 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f0178a00700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0168003f20 tx=0x7f0168011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01680038b0 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0168003a50 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741a4b30 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0168019550 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.384+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a7740 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.385+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0174064e10 con 0x7f01741107f0 2024-05-27T06:44:04.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.386+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0168019850 con 0x7f01741107f0 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.386+0000 7f0171600700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 0x7f015c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.386+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0168053d30 con 0x7f01741107f0 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.386+0000 7f0173e00700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 0x7f015c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.387+0000 7f0173e00700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 0x7f015c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f0164006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.393+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0168019c40 con 0x7f01741107f0 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.649+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0174080070 con 0x7f015c03e420 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.665+0000 7f0171600700 1 -- 172.21.15.16:0/2859312707 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0174080070 con 0x7f015c03e420 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.668+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 msgr2=0x7f015c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.668+0000 7f017b400700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 0x7f015c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f0164006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.668+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 msgr2=0x7f01741a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.668+0000 7f017b400700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0168003f20 tx=0x7f0168011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 shutdown_connections 2024-05-27T06:44:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f015c03e420 0x7f015c040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:04.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 --2- 172.21.15.16:0/2859312707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01741107f0 0x7f01741a4360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:04.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 >> 172.21.15.16:0/2859312707 conn(0x7f017407f0b0 msgr2=0x7f017407f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:04.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 shutdown_connections 2024-05-27T06:44:04.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:03.669+0000 7f017b400700 1 -- 172.21.15.16:0/2859312707 wait complete. 2024-05-27T06:44:05.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:04 smithi016 ceph-mon[31415]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:06.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:05 smithi016 ceph-mon[31415]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:06.270 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T06:44:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.064+0000 7f5dc0a00700 1 Processor -- start 2024-05-27T06:44:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.064+0000 7f5dc0a00700 1 -- start start 2024-05-27T06:44:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.065+0000 7f5dc0a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.065+0000 7f5dc0a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc10c490 con 0x7f5dbc10ba90 2024-05-27T06:44:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.065+0000 7f5dba800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.066+0000 7f5dba800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59202/0 (socket says 172.21.15.16:59202) 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.066+0000 7f5dba800700 1 -- 172.21.15.16:0/3965457752 learned_addr learned my addr 172.21.15.16:0/3965457752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.066+0000 7f5dba800700 1 -- 172.21.15.16:0/3965457752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc10cc60 con 0x7f5dbc10ba90 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.066+0000 7f5dba800700 1 --2- 172.21.15.16:0/3965457752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098a0 tx=0x7f5dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89813104dbad5e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.067+0000 7f5db9400700 1 -- 172.21.15.16:0/3965457752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac004160 con 0x7f5dbc10ba90 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.067+0000 7f5db9400700 1 -- 172.21.15.16:0/3965457752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5dac004300 con 0x7f5dbc10ba90 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.067+0000 7f5db9400700 1 -- 172.21.15.16:0/3965457752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac0119e0 con 0x7f5dbc10ba90 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 -- 172.21.15.16:0/3965457752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 msgr2=0x7f5dbc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/3965457752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098a0 tx=0x7f5dac009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 -- 172.21.15.16:0/3965457752 shutdown_connections 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/3965457752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc10bea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 -- 172.21.15.16:0/3965457752 >> 172.21.15.16:0/3965457752 conn(0x7f5dbc1074c0 msgr2=0x7f5dbc1098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 -- 172.21.15.16:0/3965457752 shutdown_connections 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.068+0000 7f5dc0a00700 1 -- 172.21.15.16:0/3965457752 wait complete. 2024-05-27T06:44:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.069+0000 7f5dc0a00700 1 Processor -- start 2024-05-27T06:44:06.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.069+0000 7f5dc0a00700 1 -- start start 2024-05-27T06:44:06.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dc0a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:06.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dc0a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc110010 con 0x7f5dbc10ba90 2024-05-27T06:44:06.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dba800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dba800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59218/0 (socket says 172.21.15.16:59218) 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dba800700 1 -- 172.21.15.16:0/755763199 learned_addr learned my addr 172.21.15.16:0/755763199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.070+0000 7f5dba800700 1 -- 172.21.15.16:0/755763199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1a0340 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5dba800700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0119b0 tx=0x7f5dac011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac003970 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5dac003b10 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac024610 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1a05d0 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.071+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a31e0 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.072+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc064e10 con 0x7f5dbc10ba90 2024-05-27T06:44:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.073+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5dac02d3f0 con 0x7f5dbc10ba90 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.073+0000 7f5db3400700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 0x7f5da00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.073+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5dac053b80 con 0x7f5dbc10ba90 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.073+0000 7f5db9e00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 0x7f5da00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.074+0000 7f5db9e00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 0x7f5da00409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.079+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5dac00e260 con 0x7f5dbc10ba90 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.317+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5dbc109480 con 0x7f5da003e4c0 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.319+0000 7f5db3400700 1 -- 172.21.15.16:0/755763199 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5dbc109480 con 0x7f5da003e4c0 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 msgr2=0x7f5da00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 0x7f5da00409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 msgr2=0x7f5dbc19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0119b0 tx=0x7f5dac011a90 comp rx=0 tx=0).stop 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 shutdown_connections 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f5da003e4c0 0x7f5da00409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:06.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 --2- 172.21.15.16:0/755763199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dbc10ba90 0x7f5dbc19fe00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:06.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 >> 172.21.15.16:0/755763199 conn(0x7f5dbc1074c0 msgr2=0x7f5dbc108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:06.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 shutdown_connections 2024-05-27T06:44:06.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:05.322+0000 7f5dc0a00700 1 -- 172.21.15.16:0/755763199 wait complete. 2024-05-27T06:44:06.280 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-05-27T06:44:07.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:07 smithi016 ceph-mon[31415]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:07.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:07 smithi016 ceph-mon[31415]: Generating ssh key... 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: Generating public/private rsa key pair. 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: Your identification has been saved in /tmp/tmp7cwbw7om/key. 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: Your public key has been saved in /tmp/tmp7cwbw7om/key.pub. 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: The key fingerprint is: 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: SHA256:x2p0oTvO2iEJx+g+mhRYiRrSvFeNIg/yf5LHkm9NTk4 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: The key's randomart image is: 2024-05-27T06:44:08.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: +---[RSA 3072]----+ 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | + . o | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: |= B . o . . | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: |o* = = o . | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: |o + = o S + | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | = * oE= | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | . B *B* | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | . o.*.*+o | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: | o..oo.+ | 2024-05-27T06:44:08.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: +----[SHA256]-----+ 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.951+0000 7fa1c9400700 1 Processor -- start 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.951+0000 7fa1c9400700 1 -- start start 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.951+0000 7fa1c9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.951+0000 7fa1c9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410a410 con 0x7fa1c4109a10 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.952+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:08.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.952+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59230/0 (socket says 172.21.15.16:59230) 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.952+0000 7fa1c2800700 1 -- 172.21.15.16:0/1334016349 learned_addr learned my addr 172.21.15.16:0/1334016349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.953+0000 7fa1c2800700 1 -- 172.21.15.16:0/1334016349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c410abe0 con 0x7fa1c4109a10 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.953+0000 7fa1c2800700 1 --2- 172.21.15.16:0/1334016349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cb706be56030 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.953+0000 7fa1c1e00700 1 -- 172.21.15.16:0/1334016349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b4004160 con 0x7fa1c4109a10 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.953+0000 7fa1c1e00700 1 -- 172.21.15.16:0/1334016349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa1b4004300 con 0x7fa1c4109a10 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.953+0000 7fa1c1e00700 1 -- 172.21.15.16:0/1334016349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b40119e0 con 0x7fa1c4109a10 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.954+0000 7fa1c9400700 1 -- 172.21.15.16:0/1334016349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 msgr2=0x7fa1c4109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.954+0000 7fa1c9400700 1 --2- 172.21.15.16:0/1334016349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:08.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.954+0000 7fa1c9400700 1 -- 172.21.15.16:0/1334016349 shutdown_connections 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.954+0000 7fa1c9400700 1 --2- 172.21.15.16:0/1334016349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4109a10 0x7fa1c4109e20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.954+0000 7fa1c9400700 1 -- 172.21.15.16:0/1334016349 >> 172.21.15.16:0/1334016349 conn(0x7fa1c4105070 msgr2=0x7fa1c4107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.955+0000 7fa1c9400700 1 -- 172.21.15.16:0/1334016349 shutdown_connections 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.955+0000 7fa1c9400700 1 -- 172.21.15.16:0/1334016349 wait complete. 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.956+0000 7fa1c9400700 1 Processor -- start 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.956+0000 7fa1c9400700 1 -- start start 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.956+0000 7fa1c9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.956+0000 7fa1c9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410df90 con 0x7fa1c419deb0 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.957+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.957+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59234/0 (socket says 172.21.15.16:59234) 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.957+0000 7fa1c2800700 1 -- 172.21.15.16:0/3625760395 learned_addr learned my addr 172.21.15.16:0/3625760395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.957+0000 7fa1c2800700 1 -- 172.21.15.16:0/3625760395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c419e860 con 0x7fa1c419deb0 2024-05-27T06:44:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.957+0000 7fa1c2800700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000c00 tx=0x7fa1b4012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.958+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b40036a0 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.958+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa1b401a440 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.958+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c419eaf0 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.958+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b401b5d0 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.958+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a15c0 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.959+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c4064e10 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.959+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa1b4003840 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.960+0000 7fa1bbe00700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 0x7fa1a80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.960+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1b4053500 con 0x7fa1c419deb0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.960+0000 7fa1b9e00700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 0x7fa1a80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:08.580 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.961+0000 7fa1b9e00700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 0x7fa1a80409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac006fd0 tx=0x7fa1ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:06.966+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1b4016260 con 0x7fa1c419deb0 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.205+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa1c4063bf0 con 0x7fa1a803e4c0 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.848+0000 7fa1bbe00700 1 -- 172.21.15.16:0/3625760395 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1c4063bf0 con 0x7fa1a803e4c0 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 msgr2=0x7fa1a80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 0x7fa1a80409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac006fd0 tx=0x7fa1ac006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 msgr2=0x7fa1c419e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000c00 tx=0x7fa1b4012f80 comp rx=0 tx=0).stop 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 shutdown_connections 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fa1a803e4c0 0x7fa1a80409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 --2- 172.21.15.16:0/3625760395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c419deb0 0x7fa1c419e2e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 >> 172.21.15.16:0/3625760395 conn(0x7fa1c4105070 msgr2=0x7fa1c4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 shutdown_connections 2024-05-27T06:44:08.582 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:07.852+0000 7fa1c9400700 1 -- 172.21.15.16:0/3625760395 wait complete. 2024-05-27T06:44:09.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:08 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:09.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:08 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:10.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:44:10.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.330+0000 7f8da6800700 1 Processor -- start 2024-05-27T06:44:10.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.330+0000 7f8da6800700 1 -- start start 2024-05-27T06:44:10.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8da6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:10.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8da6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010c600 con 0x7f8da010bc00 2024-05-27T06:44:10.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:10.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59242/0 (socket says 172.21.15.16:59242) 2024-05-27T06:44:10.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8d9f400700 1 -- 172.21.15.16:0/3344060652 learned_addr learned my addr 172.21.15.16:0/3344060652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:10.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.331+0000 7f8d9f400700 1 -- 172.21.15.16:0/3344060652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da010cdd0 con 0x7f8da010bc00 2024-05-27T06:44:10.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.332+0000 7f8d9f400700 1 --2- 172.21.15.16:0/3344060652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098a0 tx=0x7f8d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f1cf01ded642f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:10.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.332+0000 7f8d9ea00700 1 -- 172.21.15.16:0/3344060652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d90004160 con 0x7f8da010bc00 2024-05-27T06:44:10.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.332+0000 7f8d9ea00700 1 -- 172.21.15.16:0/3344060652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8d90004300 con 0x7f8da010bc00 2024-05-27T06:44:10.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.332+0000 7f8d9ea00700 1 -- 172.21.15.16:0/3344060652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d900119e0 con 0x7f8da010bc00 2024-05-27T06:44:10.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 -- 172.21.15.16:0/3344060652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 msgr2=0x7f8da010c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:10.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 --2- 172.21.15.16:0/3344060652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098a0 tx=0x7f8d90009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:10.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 -- 172.21.15.16:0/3344060652 shutdown_connections 2024-05-27T06:44:10.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 --2- 172.21.15.16:0/3344060652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da010c010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:10.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 -- 172.21.15.16:0/3344060652 >> 172.21.15.16:0/3344060652 conn(0x7f8da01074c0 msgr2=0x7f8da01098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:10.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 -- 172.21.15.16:0/3344060652 shutdown_connections 2024-05-27T06:44:10.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.333+0000 7f8da6800700 1 -- 172.21.15.16:0/3344060652 wait complete. 2024-05-27T06:44:10.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.334+0000 7f8da6800700 1 Processor -- start 2024-05-27T06:44:10.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8da6800700 1 -- start start 2024-05-27T06:44:10.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8da6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:10.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8da6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0110180 con 0x7f8da010bc00 2024-05-27T06:44:10.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:10.199 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59258/0 (socket says 172.21.15.16:59258) 2024-05-27T06:44:10.199 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.335+0000 7f8d9f400700 1 -- 172.21.15.16:0/177241161 learned_addr learned my addr 172.21.15.16:0/177241161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:10.199 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8d9f400700 1 -- 172.21.15.16:0/177241161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da01a04c0 con 0x7f8da010bc00 2024-05-27T06:44:10.200 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8d9f400700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009fb0 tx=0x7f8d90011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:10.200 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d900038b0 con 0x7f8da010bc00 2024-05-27T06:44:10.200 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8d90003a50 con 0x7f8da010bc00 2024-05-27T06:44:10.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da01a0750 con 0x7f8da010bc00 2024-05-27T06:44:10.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d9001a5c0 con 0x7f8da010bc00 2024-05-27T06:44:10.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.336+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da01a3360 con 0x7f8da010bc00 2024-05-27T06:44:10.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.337+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da0064e10 con 0x7f8da010bc00 2024-05-27T06:44:10.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.338+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8d9001a8c0 con 0x7f8da010bc00 2024-05-27T06:44:10.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.338+0000 7f8d9cc00700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 0x7f8d84040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:10.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.338+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d90053bf0 con 0x7f8da010bc00 2024-05-27T06:44:10.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.339+0000 7f8d97e00700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 0x7f8d84040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:10.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.340+0000 7f8d97e00700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 0x7f8d84040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d88006fd0 tx=0x7f8d88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:10.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.346+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d90017e90 con 0x7f8da010bc00 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.583+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8da01093e0 con 0x7f8d8403e420 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.585+0000 7f8d9cc00700 1 -- 172.21.15.16:0/177241161 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8da01093e0 con 0x7f8d8403e420 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 msgr2=0x7f8d84040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 0x7f8d84040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d88006fd0 tx=0x7f8d88006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 msgr2=0x7f8da019ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009fb0 tx=0x7f8d90011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 shutdown_connections 2024-05-27T06:44:10.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f8d8403e420 0x7f8d84040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:10.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 --2- 172.21.15.16:0/177241161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da010bc00 0x7f8da019ff80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:10.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.588+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 >> 172.21.15.16:0/177241161 conn(0x7f8da01074c0 msgr2=0x7f8da0108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:10.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.589+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 shutdown_connections 2024-05-27T06:44:10.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:09.589+0000 7f8da6800700 1 -- 172.21.15.16:0/177241161 wait complete. 2024-05-27T06:44:10.206 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T06:44:10.206 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T06:44:10.208 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-05-27T06:44:10.785 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:10 smithi016 ceph-mon[31415]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:11.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:11 smithi016 ceph-mon[31415]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:13.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:13 smithi016 ceph-mon[31415]: Deploying cephadm binary to smithi016 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.951+0000 7f1f81600700 1 Processor -- start 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.952+0000 7f1f81600700 1 -- start start 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.952+0000 7f1f81600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.952+0000 7f1f81600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c10c600 con 0x7f1f7c10bc00 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.952+0000 7f1f7aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.953+0000 7f1f7aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59270/0 (socket says 172.21.15.16:59270) 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.953+0000 7f1f7aa00700 1 -- 172.21.15.16:0/2372731696 learned_addr learned my addr 172.21.15.16:0/2372731696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:18.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.953+0000 7f1f7aa00700 1 -- 172.21.15.16:0/2372731696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c10cdd0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.953+0000 7f1f7aa00700 1 --2- 172.21.15.16:0/2372731696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0098a0 tx=0x7f1f6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cee5844b42e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.954+0000 7f1f79600700 1 -- 172.21.15.16:0/2372731696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c004160 con 0x7f1f7c10bc00 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.954+0000 7f1f79600700 1 -- 172.21.15.16:0/2372731696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1f6c004300 con 0x7f1f7c10bc00 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.954+0000 7f1f79600700 1 -- 172.21.15.16:0/2372731696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c0119e0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 -- 172.21.15.16:0/2372731696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 msgr2=0x7f1f7c10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 --2- 172.21.15.16:0/2372731696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0098a0 tx=0x7f1f6c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 -- 172.21.15.16:0/2372731696 shutdown_connections 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 --2- 172.21.15.16:0/2372731696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c10c010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 -- 172.21.15.16:0/2372731696 >> 172.21.15.16:0/2372731696 conn(0x7f1f7c1074c0 msgr2=0x7f1f7c1098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 -- 172.21.15.16:0/2372731696 shutdown_connections 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.955+0000 7f1f81600700 1 -- 172.21.15.16:0/2372731696 wait complete. 2024-05-27T06:44:18.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.956+0000 7f1f81600700 1 Processor -- start 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f81600700 1 -- start start 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f81600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f81600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c110180 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f7aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f7aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59274/0 (socket says 172.21.15.16:59274) 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.957+0000 7f1f7aa00700 1 -- 172.21.15.16:0/2876489070 learned_addr learned my addr 172.21.15.16:0/2876489070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f7aa00700 1 -- 172.21.15.16:0/2876489070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c1a04c0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f7aa00700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0046e0 tx=0x7f1f6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c0039e0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1f6c003b80 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c024700 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.958+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c1a0750 con 0x7f1f7c10bc00 2024-05-27T06:44:18.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.959+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c1a3360 con 0x7f1f7c10bc00 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.960+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c064e10 con 0x7f1f7c10bc00 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.960+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1f6c022020 con 0x7f1f7c10bc00 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.961+0000 7f1f73400700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 0x7f1f600409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.961+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f6c0539b0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.961+0000 7f1f7a000700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 0x7f1f600409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.962+0000 7f1f7a000700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 0x7f1f600409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f68006fd0 tx=0x7f1f68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:10.966+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f6c00b3e0 con 0x7f1f7c10bc00 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:11.207+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7f1f7c1093e0 con 0x7f1f6003e4c0 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.659+0000 7f1f73400700 1 -- 172.21.15.16:0/2876489070 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1f7c1093e0 con 0x7f1f6003e4c0 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.664+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 msgr2=0x7f1f600409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.664+0000 7f1f81600700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 0x7f1f600409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f68006fd0 tx=0x7f1f68006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.664+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 msgr2=0x7f1f7c19ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:18.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.664+0000 7f1f81600700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0046e0 tx=0x7f1f6c011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 shutdown_connections 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f1f6003e4c0 0x7f1f600409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 --2- 172.21.15.16:0/2876489070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7c10bc00 0x7f1f7c19ff80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 >> 172.21.15.16:0/2876489070 conn(0x7f1f7c1074c0 msgr2=0x7f1f7c108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 shutdown_connections 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:17.665+0000 7f1f81600700 1 -- 172.21.15.16:0/2876489070 wait complete. 2024-05-27T06:44:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-05-27T06:44:18.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:18 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:18.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:18 smithi016 ceph-mon[31415]: Added host smithi016 2024-05-27T06:44:18.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:18 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:20.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.132+0000 7f0361c00700 1 Processor -- start 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.132+0000 7f0361c00700 1 -- start start 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.132+0000 7f0361c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.132+0000 7f0361c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f035c10c490 con 0x7f035c10ba90 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.133+0000 7f035be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.133+0000 7f035be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45142/0 (socket says 172.21.15.16:45142) 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.133+0000 7f035be00700 1 -- 172.21.15.16:0/1027058983 learned_addr learned my addr 172.21.15.16:0/1027058983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.134+0000 7f035be00700 1 -- 172.21.15.16:0/1027058983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c10cc60 con 0x7f035c10ba90 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.134+0000 7f035be00700 1 --2- 172.21.15.16:0/1027058983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1e10d4174f0211 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.134+0000 7f035aa00700 1 -- 172.21.15.16:0/1027058983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c004160 con 0x7f035c10ba90 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f035aa00700 1 -- 172.21.15.16:0/1027058983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f034c004300 con 0x7f035c10ba90 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f035aa00700 1 -- 172.21.15.16:0/1027058983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c0119e0 con 0x7f035c10ba90 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f0361c00700 1 -- 172.21.15.16:0/1027058983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 msgr2=0x7f035c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f0361c00700 1 --2- 172.21.15.16:0/1027058983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f0361c00700 1 -- 172.21.15.16:0/1027058983 shutdown_connections 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f0361c00700 1 --2- 172.21.15.16:0/1027058983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c10bea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.135+0000 7f0361c00700 1 -- 172.21.15.16:0/1027058983 >> 172.21.15.16:0/1027058983 conn(0x7f035c1074c0 msgr2=0x7f035c1098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.136+0000 7f0361c00700 1 -- 172.21.15.16:0/1027058983 shutdown_connections 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.136+0000 7f0361c00700 1 -- 172.21.15.16:0/1027058983 wait complete. 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f0361c00700 1 Processor -- start 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f0361c00700 1 -- start start 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f0361c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f0361c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f035c110010 con 0x7f035c10ba90 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f035be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f035be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45144/0 (socket says 172.21.15.16:45144) 2024-05-27T06:44:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.137+0000 7f035be00700 1 -- 172.21.15.16:0/2543788105 learned_addr learned my addr 172.21.15.16:0/2543788105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f035be00700 1 -- 172.21.15.16:0/2543788105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c1a0340 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f035be00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f034c009fb0 tx=0x7f034c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c0038b0 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f034c003a50 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c1a05d0 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.138+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c019550 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.139+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c1a31e0 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.140+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c064e10 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.140+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f034c019800 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.140+0000 7f0358c00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 0x7f0340040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.141+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f034c053b40 con 0x7f035c10ba90 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.141+0000 7f035b400700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 0x7f0340040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:20.087 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.141+0000 7f035b400700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 0x7f0340040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.147+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f034c017e90 con 0x7f035c10ba90 2024-05-27T06:44:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.399+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f035c109480 con 0x7f034003e470 2024-05-27T06:44:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.410+0000 7f0358c00700 1 -- 172.21.15.16:0/2543788105 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f035c109480 con 0x7f034003e470 2024-05-27T06:44:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.413+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 msgr2=0x7f0340040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.413+0000 7f0361c00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 0x7f0340040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.413+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 msgr2=0x7f035c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.413+0000 7f0361c00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f034c009fb0 tx=0x7f034c011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 shutdown_connections 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f034003e470 0x7f0340040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 --2- 172.21.15.16:0/2543788105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f035c10ba90 0x7f035c19fe00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 >> 172.21.15.16:0/2543788105 conn(0x7f035c1074c0 msgr2=0x7f035c108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 shutdown_connections 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:19.414+0000 7f0361c00700 1 -- 172.21.15.16:0/2543788105 wait complete. 2024-05-27T06:44:20.090 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-05-27T06:44:20.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:20 smithi016 ceph-mon[31415]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:20.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:20 smithi016 ceph-mon[31415]: Saving service mon spec with placement count:5 2024-05-27T06:44:20.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:20 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:22.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:22 smithi016 ceph-mon[31415]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:44:22.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:22 smithi016 ceph-mon[31415]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:22.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:22 smithi016 ceph-mon[31415]: Saving service mgr spec with placement count:2 2024-05-27T06:44:22.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:22 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:22.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T06:44:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.926+0000 7f4478a00700 1 Processor -- start 2024-05-27T06:44:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.926+0000 7f4478a00700 1 -- start start 2024-05-27T06:44:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.927+0000 7f4478a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.927+0000 7f4478a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447407d8b0 con 0x7f447410a4f0 2024-05-27T06:44:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.927+0000 7f4471e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.927+0000 7f4471e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45154/0 (socket says 172.21.15.16:45154) 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.927+0000 7f4471e00700 1 -- 172.21.15.16:0/119884846 learned_addr learned my addr 172.21.15.16:0/119884846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.928+0000 7f4471e00700 1 -- 172.21.15.16:0/119884846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447410ce50 con 0x7f447410a4f0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.928+0000 7f4471e00700 1 --2- 172.21.15.16:0/119884846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44640098a0 tx=0x7f4464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50b2afa0c3c27c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.928+0000 7f4470a00700 1 -- 172.21.15.16:0/119884846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4464004160 con 0x7f447410a4f0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.928+0000 7f4470a00700 1 -- 172.21.15.16:0/119884846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4464004300 con 0x7f447410a4f0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.928+0000 7f4470a00700 1 -- 172.21.15.16:0/119884846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44640119e0 con 0x7f447410a4f0 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.929+0000 7f4478a00700 1 -- 172.21.15.16:0/119884846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 msgr2=0x7f447410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.929+0000 7f4478a00700 1 --2- 172.21.15.16:0/119884846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44640098a0 tx=0x7f4464009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.929+0000 7f4478a00700 1 -- 172.21.15.16:0/119884846 shutdown_connections 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.929+0000 7f4478a00700 1 --2- 172.21.15.16:0/119884846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447410c910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.929+0000 7f4478a00700 1 -- 172.21.15.16:0/119884846 >> 172.21.15.16:0/119884846 conn(0x7f4474103f50 msgr2=0x7f4474106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.930+0000 7f4478a00700 1 -- 172.21.15.16:0/119884846 shutdown_connections 2024-05-27T06:44:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.930+0000 7f4478a00700 1 -- 172.21.15.16:0/119884846 wait complete. 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.931+0000 7f4478a00700 1 Processor -- start 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.931+0000 7f4478a00700 1 -- start start 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.931+0000 7f4478a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.931+0000 7f4478a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447410ddb0 con 0x7f447410a4f0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.932+0000 7f4471e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.932+0000 7f4471e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45156/0 (socket says 172.21.15.16:45156) 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.932+0000 7f4471e00700 1 -- 172.21.15.16:0/2382971190 learned_addr learned my addr 172.21.15.16:0/2382971190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.932+0000 7f4471e00700 1 -- 172.21.15.16:0/2382971190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44741a0450 con 0x7f447410a4f0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.932+0000 7f4471e00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44640116a0 tx=0x7f4464011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.933+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4464003970 con 0x7f447410a4f0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.933+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4464003b10 con 0x7f447410a4f0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.933+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4464019550 con 0x7f447410a4f0 2024-05-27T06:44:22.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.933+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44741a06e0 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.933+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44741a32f0 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.934+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4474064e10 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.934+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f446401a430 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.935+0000 7f446aa00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 0x7f44580409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.935+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4464053970 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.936+0000 7f4471400700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 0x7f44580409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.938+0000 7f4471400700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 0x7f44580409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4460006fd0 tx=0x7f4460006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:20.941+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f446401a720 con 0x7f447410a4f0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.204+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4474108ab0 con 0x7f445803e4c0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.212+0000 7f446aa00700 1 -- 172.21.15.16:0/2382971190 <== mgr.14120 v2:172.21.15.16:6800/2390178341 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4474108ab0 con 0x7f445803e4c0 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 msgr2=0x7f44580409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 0x7f44580409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4460006fd0 tx=0x7f4460006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 msgr2=0x7f447419ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:22.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44640116a0 tx=0x7f4464011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 shutdown_connections 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f445803e4c0 0x7f44580409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 --2- 172.21.15.16:0/2382971190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447410a4f0 0x7f447419ff10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 >> 172.21.15.16:0/2382971190 conn(0x7f4474103f50 msgr2=0x7f44741083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 shutdown_connections 2024-05-27T06:44:22.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:21.215+0000 7f4478a00700 1 -- 172.21.15.16:0/2382971190 wait complete. 2024-05-27T06:44:24.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:24 smithi016 ceph-mon[31415]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:44:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:24 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:24 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4273189000' entity='client.admin' 2024-05-27T06:44:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:24 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.547+0000 7f86e8000700 1 Processor -- start 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.547+0000 7f86e8000700 1 -- start start 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.548+0000 7f86e8000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.548+0000 7f86e8000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86e010a450 con 0x7f86e0109a50 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.548+0000 7f86e5600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.548+0000 7f86e5600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41428/0 (socket says 172.21.15.16:41428) 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.548+0000 7f86e5600700 1 -- 172.21.15.16:0/261492688 learned_addr learned my addr 172.21.15.16:0/261492688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.549+0000 7f86e5600700 1 -- 172.21.15.16:0/261492688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e010ac20 con 0x7f86e0109a50 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.549+0000 7f86e5600700 1 --2- 172.21.15.16:0/261492688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86d40098a0 tx=0x7f86d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef79b2297f5c8446 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.550+0000 7f86dfe00700 1 -- 172.21.15.16:0/261492688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d4004160 con 0x7f86e0109a50 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.550+0000 7f86dfe00700 1 -- 172.21.15.16:0/261492688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d4004300 con 0x7f86e0109a50 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.550+0000 7f86dfe00700 1 -- 172.21.15.16:0/261492688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d4011960 con 0x7f86e0109a50 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 -- 172.21.15.16:0/261492688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 msgr2=0x7f86e0109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 --2- 172.21.15.16:0/261492688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86d40098a0 tx=0x7f86d4009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 -- 172.21.15.16:0/261492688 shutdown_connections 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 --2- 172.21.15.16:0/261492688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e0109e60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 -- 172.21.15.16:0/261492688 >> 172.21.15.16:0/261492688 conn(0x7f86e01050b0 msgr2=0x7f86e0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 -- 172.21.15.16:0/261492688 shutdown_connections 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.551+0000 7f86e8000700 1 -- 172.21.15.16:0/261492688 wait complete. 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.553+0000 7f86e8000700 1 Processor -- start 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.553+0000 7f86e8000700 1 -- start start 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.554+0000 7f86e8000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.554+0000 7f86e8000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86e010dfd0 con 0x7f86e0109a50 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.554+0000 7f86e5600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:24.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.554+0000 7f86e5600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41432/0 (socket says 172.21.15.16:41432) 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.554+0000 7f86e5600700 1 -- 172.21.15.16:0/4273189000 learned_addr learned my addr 172.21.15.16:0/4273189000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86e5600700 1 -- 172.21.15.16:0/4273189000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e007efb0 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86e5600700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f86d4011980 tx=0x7f86d4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d401f070 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d4003aa0 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d4019560 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e007d200 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.555+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e007d6e0 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.557+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0064e10 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.558+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f86d4022070 con 0x7f86e0109a50 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.558+0000 7f86de000700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 0x7f86c80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:24.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.559+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86d40539c0 con 0x7f86e0109a50 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.559+0000 7f86e4c00700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 0x7f86c80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.560+0000 7f86e4c00700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 0x7f86c80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.565+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86d4053350 con 0x7f86e0109a50 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.805+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f86e007dd80 con 0x7f86e0109a50 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.810+0000 7f86de000700 1 -- 172.21.15.16:0/4273189000 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f86d4023d60 con 0x7f86e0109a50 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 msgr2=0x7f86c80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 0x7f86c80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 msgr2=0x7f86e007ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f86d4011980 tx=0x7f86d4011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 shutdown_connections 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f86c803e4c0 0x7f86c80409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 --2- 172.21.15.16:0/4273189000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0109a50 0x7f86e007ea70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 >> 172.21.15.16:0/4273189000 conn(0x7f86e01050b0 msgr2=0x7f86e0105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:24.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 shutdown_connections 2024-05-27T06:44:24.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:23.814+0000 7f86e8000700 1 -- 172.21.15.16:0/4273189000 wait complete. 2024-05-27T06:44:26.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:26 smithi016 ceph-mon[31415]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:44:26.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:26 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2798273527' entity='client.admin' 2024-05-27T06:44:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.363+0000 7f06ed800700 1 Processor -- start 2024-05-27T06:44:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.363+0000 7f06ed800700 1 -- start start 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.364+0000 7f06ed800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.364+0000 7f06ed800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8113200 con 0x7f06e81107f0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.364+0000 7f06e6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.364+0000 7f06e6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41442/0 (socket says 172.21.15.16:41442) 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.364+0000 7f06e6a00700 1 -- 172.21.15.16:0/3861759850 learned_addr learned my addr 172.21.15.16:0/3861759850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.365+0000 7f06e6a00700 1 -- 172.21.15.16:0/3861759850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e8113380 con 0x7f06e81107f0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.365+0000 7f06e6a00700 1 --2- 172.21.15.16:0/3861759850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06d80098a0 tx=0x7f06d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddfb84e9dfd1aba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.365+0000 7f06e5600700 1 -- 172.21.15.16:0/3861759850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d8004160 con 0x7f06e81107f0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.365+0000 7f06e5600700 1 -- 172.21.15.16:0/3861759850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06d8004300 con 0x7f06e81107f0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.365+0000 7f06e5600700 1 -- 172.21.15.16:0/3861759850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d8011960 con 0x7f06e81107f0 2024-05-27T06:44:26.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 -- 172.21.15.16:0/3861759850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 msgr2=0x7f06e8112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 --2- 172.21.15.16:0/3861759850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06d80098a0 tx=0x7f06d8009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 -- 172.21.15.16:0/3861759850 shutdown_connections 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 --2- 172.21.15.16:0/3861759850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e8112c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 -- 172.21.15.16:0/3861759850 >> 172.21.15.16:0/3861759850 conn(0x7f06e807f0b0 msgr2=0x7f06e8081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 -- 172.21.15.16:0/3861759850 shutdown_connections 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.367+0000 7f06ed800700 1 -- 172.21.15.16:0/3861759850 wait complete. 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.368+0000 7f06ed800700 1 Processor -- start 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06ed800700 1 -- start start 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06ed800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06ed800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81143e0 con 0x7f06e81107f0 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06e6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06e6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41452/0 (socket says 172.21.15.16:41452) 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.369+0000 7f06e6a00700 1 -- 172.21.15.16:0/2798273527 learned_addr learned my addr 172.21.15.16:0/2798273527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.370+0000 7f06e6a00700 1 -- 172.21.15.16:0/2798273527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e81a48c0 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.370+0000 7f06e6a00700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06d8011670 tx=0x7f06d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.370+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d801f070 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.371+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06d8003b20 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.371+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d8019580 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.371+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a4b50 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.371+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a7760 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.372+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e8064e10 con 0x7f06e81107f0 2024-05-27T06:44:26.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.372+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f06d8022070 con 0x7f06e81107f0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.373+0000 7f06df400700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 0x7f06cc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.373+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f06d8053ac0 con 0x7f06e81107f0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.373+0000 7f06e6000700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 0x7f06cc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.374+0000 7f06e6000700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 0x7f06cc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06d4006fd0 tx=0x7f06d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.379+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06d8053410 con 0x7f06e81107f0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.618+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f06e81a7d80 con 0x7f06e81107f0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.621+0000 7f06df400700 1 -- 172.21.15.16:0/2798273527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f06d802cd00 con 0x7f06e81107f0 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 msgr2=0x7f06cc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 0x7f06cc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06d4006fd0 tx=0x7f06d4006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:26.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 msgr2=0x7f06e81a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06d8011670 tx=0x7f06d8011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 shutdown_connections 2024-05-27T06:44:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f06cc03e4c0 0x7f06cc0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 --2- 172.21.15.16:0/2798273527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e81107f0 0x7f06e81a4380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.625+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 >> 172.21.15.16:0/2798273527 conn(0x7f06e807f0b0 msgr2=0x7f06e807f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.626+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 shutdown_connections 2024-05-27T06:44:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:25.626+0000 7f06ed800700 1 -- 172.21.15.16:0/2798273527 wait complete. 2024-05-27T06:44:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-05-27T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:28 smithi016 ceph-mon[31415]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:28 smithi016 ceph-mon[31415]: from='mgr.14120 172.21.15.16:0/2830969700' entity='mgr.x' 2024-05-27T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:28 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4259830101' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T06:44:28.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setuser ceph since I am not root 2024-05-27T06:44:28.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setgroup ceph since I am not root 2024-05-27T06:44:28.676 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:28.430+0000 7fdecd68a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T06:44:28.676 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:28.681+0000 7fdecd68a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:44:28.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.523+0000 7f7b25800700 1 Processor -- start 2024-05-27T06:44:28.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.523+0000 7f7b25800700 1 -- start start 2024-05-27T06:44:28.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.524+0000 7f7b25800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.524+0000 7f7b25800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2007e100 con 0x7f7b2007f700 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.524+0000 7f7b1ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.524+0000 7f7b1ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41462/0 (socket says 172.21.15.16:41462) 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.524+0000 7f7b1ea00700 1 -- 172.21.15.16:0/3478089999 learned_addr learned my addr 172.21.15.16:0/3478089999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.525+0000 7f7b1ea00700 1 -- 172.21.15.16:0/3478089999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2007e280 con 0x7f7b2007f700 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.525+0000 7f7b1ea00700 1 --2- 172.21.15.16:0/3478089999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b100098a0 tx=0x7f7b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0da75152912fdca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.526+0000 7f7b1d600700 1 -- 172.21.15.16:0/3478089999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b10004160 con 0x7f7b2007f700 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.526+0000 7f7b1d600700 1 -- 172.21.15.16:0/3478089999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b10004300 con 0x7f7b2007f700 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.526+0000 7f7b1d600700 1 -- 172.21.15.16:0/3478089999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b10011960 con 0x7f7b2007f700 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 -- 172.21.15.16:0/3478089999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 msgr2=0x7f7b2007db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 --2- 172.21.15.16:0/3478089999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b100098a0 tx=0x7f7b10009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 -- 172.21.15.16:0/3478089999 shutdown_connections 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 --2- 172.21.15.16:0/3478089999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2007db30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 -- 172.21.15.16:0/3478089999 >> 172.21.15.16:0/3478089999 conn(0x7f7b20105090 msgr2=0x7f7b20107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 -- 172.21.15.16:0/3478089999 shutdown_connections 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.527+0000 7f7b25800700 1 -- 172.21.15.16:0/3478089999 wait complete. 2024-05-27T06:44:28.849 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.529+0000 7f7b25800700 1 Processor -- start 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.529+0000 7f7b25800700 1 -- start start 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.530+0000 7f7b25800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.530+0000 7f7b25800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20110050 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.530+0000 7f7b1ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.530+0000 7f7b1ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41474/0 (socket says 172.21.15.16:41474) 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.530+0000 7f7b1ea00700 1 -- 172.21.15.16:0/4259830101 learned_addr learned my addr 172.21.15.16:0/4259830101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b1ea00700 1 -- 172.21.15.16:0/4259830101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b201a04a0 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b1ea00700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009520 tx=0x7f7b10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b10003aa0 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b1001b8d0 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b10023790 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.531+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b201a0730 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.532+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b201a3340 con 0x7f7b2007f700 2024-05-27T06:44:28.850 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.533+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b20064e10 con 0x7f7b2007f700 2024-05-27T06:44:28.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.534+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7b1001b440 con 0x7f7b2007f700 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.534+0000 7f7b17400700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 0x7f7b040409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.534+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b100542c0 con 0x7f7b2007f700 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.535+0000 7f7b1e000700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 0x7f7b040409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.536+0000 7f7b1e000700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 0x7f7b040409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.540+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b10028060 con 0x7f7b2007f700 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:27.870+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7b201a3510 con 0x7f7b2007f700 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.023+0000 7f7b17400700 1 -- 172.21.15.16:0/4259830101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7b1001f030 con 0x7f7b2007f700 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.027+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 msgr2=0x7f7b040409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.027+0000 7f7b25800700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 0x7f7b040409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.027+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 msgr2=0x7f7b2019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.027+0000 7f7b25800700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009520 tx=0x7f7b10011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.028+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 shutdown_connections 2024-05-27T06:44:28.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.028+0000 7f7b25800700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f7b0403e4c0 0x7f7b040409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:28.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.028+0000 7f7b25800700 1 --2- 172.21.15.16:0/4259830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2007f700 0x7f7b2019ff60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:28.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.028+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 >> 172.21.15.16:0/4259830101 conn(0x7f7b20105090 msgr2=0x7f7b201058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:28.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.029+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 shutdown_connections 2024-05-27T06:44:28.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:28.029+0000 7f7b25800700 1 -- 172.21.15.16:0/4259830101 wait complete. 2024-05-27T06:44:29.010 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:28.819+0000 7fdecd68a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T06:44:29.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:29 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4259830101' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T06:44:29.264 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:29 smithi016 ceph-mon[31415]: mgrmap e9: x(active, since 27s) 2024-05-27T06:44:29.264 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:29.251+0000 7fdecd68a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T06:44:29.603 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:29.373+0000 7fdecd68a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T06:44:30.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:30 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3621962022' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T06:44:30.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:44:30.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T06:44:30.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T06:44:30.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T06:44:30.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.593+0000 7f4559000700 1 Processor -- start 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.593+0000 7f4559000700 1 -- start start 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.593+0000 7f4559000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.593+0000 7f4559000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f455407d780 con 0x7f455407ee10 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.594+0000 7f4551e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.594+0000 7f4551e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41502/0 (socket says 172.21.15.16:41502) 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.594+0000 7f4551e00700 1 -- 172.21.15.16:0/1105994308 learned_addr learned my addr 172.21.15.16:0/1105994308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.594+0000 7f4551e00700 1 -- 172.21.15.16:0/1105994308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455407d900 con 0x7f455407ee10 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.595+0000 7f4551e00700 1 --2- 172.21.15.16:0/1105994308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369ade39087c22c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:30.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.595+0000 7f4550a00700 1 -- 172.21.15.16:0/1105994308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544004160 con 0x7f455407ee10 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.595+0000 7f4550a00700 1 -- 172.21.15.16:0/1105994308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4544004300 con 0x7f455407ee10 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.595+0000 7f4550a00700 1 -- 172.21.15.16:0/1105994308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544011960 con 0x7f455407ee10 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.596+0000 7f4559000700 1 -- 172.21.15.16:0/1105994308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 msgr2=0x7f455407d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.596+0000 7f4559000700 1 --2- 172.21.15.16:0/1105994308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.597+0000 7f4559000700 1 -- 172.21.15.16:0/1105994308 shutdown_connections 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.597+0000 7f4559000700 1 --2- 172.21.15.16:0/1105994308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455407d240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.597+0000 7f4559000700 1 -- 172.21.15.16:0/1105994308 >> 172.21.15.16:0/1105994308 conn(0x7f45541050b0 msgr2=0x7f4554107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.597+0000 7f4559000700 1 -- 172.21.15.16:0/1105994308 shutdown_connections 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.597+0000 7f4559000700 1 -- 172.21.15.16:0/1105994308 wait complete. 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.598+0000 7f4559000700 1 Processor -- start 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.598+0000 7f4559000700 1 -- start start 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4559000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:30.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4559000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4554110010 con 0x7f455407ee10 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4551e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4551e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41506/0 (socket says 172.21.15.16:41506) 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4551e00700 1 -- 172.21.15.16:0/3621962022 learned_addr learned my addr 172.21.15.16:0/3621962022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.599+0000 7f4551e00700 1 -- 172.21.15.16:0/3621962022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45541a04d0 con 0x7f455407ee10 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f4551e00700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4544011670 tx=0x7f4544011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f454401f070 con 0x7f455407ee10 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45541a0760 con 0x7f455407ee10 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4544003b20 con 0x7f455407ee10 2024-05-27T06:44:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544019580 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.600+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a3370 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4544022070 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4554064e10 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f454aa00700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 0x7f4538040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4544053ab0 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f4551400700 1 -- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 msgr2=0x7f4538040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.602+0000 7f4551400700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 0x7f4538040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.609+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45440172b0 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.803+0000 7f4551400700 1 -- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 msgr2=0x7f4538040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.803+0000 7f4551400700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 0x7f4538040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.901+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f45541a3660 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.901+0000 7f454aa00700 1 -- 172.21.15.16:0/3621962022 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f45440197f0 con 0x7f455407ee10 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.904+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 msgr2=0x7f4538040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T06:44:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.904+0000 7f4559000700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 0x7f4538040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.904+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 msgr2=0x7f455419ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.904+0000 7f4559000700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4544011670 tx=0x7f4544011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.905+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 shutdown_connections 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.905+0000 7f4559000700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7f453803e520 0x7f4538040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.905+0000 7f4559000700 1 --2- 172.21.15.16:0/3621962022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f455407ee10 0x7f455419ff90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.905+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 >> 172.21.15.16:0/3621962022 conn(0x7f45541050b0 msgr2=0x7f4554105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.906+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 shutdown_connections 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:29.906+0000 7f4559000700 1 -- 172.21.15.16:0/3621962022 wait complete. 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-05-27T06:44:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-05-27T06:44:31.263 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:31.023+0000 7fdecd68a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T06:44:31.263 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:31.269+0000 7fdecd68a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T06:44:31.522 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:31.494+0000 7fdecd68a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T06:44:32.287 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:31.894+0000 7fdecd68a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T06:44:32.287 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:32.026+0000 7fdecd68a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T06:44:32.287 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:32.292+0000 7fdecd68a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T06:44:32.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:32.567+0000 7fdecd68a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:44:33.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:33.406+0000 7fdecd68a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T06:44:34.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:33.708+0000 7fdecd68a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T06:44:34.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:34 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:34.296+0000 7fdecd68a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T06:44:34.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:34 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:34.432+0000 7fdecd68a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T06:44:35.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:34 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:34.913+0000 7fdecd68a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T06:44:35.859 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:35 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:35.392+0000 7fdecd68a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T06:44:35.859 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:35 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:35.525+0000 7fdecd68a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T06:44:35.859 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:35 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:35.864+0000 7fdecd68a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T06:44:36.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:36.052+0000 7fdecd68a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T06:44:36.611 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:36.190+0000 7fdecd68a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T06:44:37.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:36.632+0000 7fdecd68a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T06:44:37.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:36.761+0000 7fdecd68a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T06:44:37.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:37.567+0000 7fdecd68a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:44:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:44:37.871+0000 7fdecd68a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: Active manager daemon x restarted 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: Activating manager daemon x 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: mgrmap e10: x(active, starting, since 0.0124376s) 2024-05-27T06:44:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:44:38.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T06:44:38.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T06:44:38.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T06:44:38.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T06:44:38.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:37 smithi016 ceph-mon[31415]: Manager daemon x is now available 2024-05-27T06:44:39.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:39.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T06:44:39.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T06:44:39.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:38 smithi016 ceph-mon[31415]: mgrmap e11: x(active, since 1.01211s) 2024-05-27T06:44:39.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.317+0000 7fedcc200700 1 Processor -- start 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.317+0000 7fedcc200700 1 -- start start 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.318+0000 7fedcc200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.318+0000 7fedcc200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410a480 con 0x7fedc4109a80 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.318+0000 7fedc9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.319+0000 7fedc9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41520/0 (socket says 172.21.15.16:41520) 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.319+0000 7fedc9800700 1 -- 172.21.15.16:0/2525805458 learned_addr learned my addr 172.21.15.16:0/2525805458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.319+0000 7fedc9800700 1 -- 172.21.15.16:0/2525805458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc410ac50 con 0x7fedc4109a80 2024-05-27T06:44:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.319+0000 7fedc9800700 1 --2- 172.21.15.16:0/2525805458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b920c0e3474742 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:39.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.320+0000 7fedc3e00700 1 -- 172.21.15.16:0/2525805458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8004160 con 0x7fedc4109a80 2024-05-27T06:44:39.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.320+0000 7fedc3e00700 1 -- 172.21.15.16:0/2525805458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedb8004300 con 0x7fedc4109a80 2024-05-27T06:44:39.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.320+0000 7fedc3e00700 1 -- 172.21.15.16:0/2525805458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8011960 con 0x7fedc4109a80 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 -- 172.21.15.16:0/2525805458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 msgr2=0x7fedc4109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 --2- 172.21.15.16:0/2525805458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 -- 172.21.15.16:0/2525805458 shutdown_connections 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 --2- 172.21.15.16:0/2525805458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc4109e90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 -- 172.21.15.16:0/2525805458 >> 172.21.15.16:0/2525805458 conn(0x7fedc41050c0 msgr2=0x7fedc4107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 -- 172.21.15.16:0/2525805458 shutdown_connections 2024-05-27T06:44:39.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.321+0000 7fedcc200700 1 -- 172.21.15.16:0/2525805458 wait complete. 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.322+0000 7fedcc200700 1 Processor -- start 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedcc200700 1 -- start start 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedcc200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedcc200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410e000 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedc9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedc9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41524/0 (socket says 172.21.15.16:41524) 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.323+0000 7fedc9800700 1 -- 172.21.15.16:0/3350039349 learned_addr learned my addr 172.21.15.16:0/3350039349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedc9800700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc41a0600 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedc9800700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedb8006af0 tx=0x7fedb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8003af0 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedb801b8d0 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc41a0890 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.324+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8023820 con 0x7fedc4109a80 2024-05-27T06:44:39.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.325+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc41a34a0 con 0x7fedc4109a80 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.326+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fedb802c3f0 con 0x7fedc4109a80 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.326+0000 7fedc2000700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.326+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fedac041140 con 0x7fedac03e520 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.326+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fedb8053f10 con 0x7fedc4109a80 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.327+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.327+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.528+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.528+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.929+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:31.929+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:32.730+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:32.730+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T06:44:39.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:34.332+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:34.332+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:37.535+0000 7fedc8e00700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2390178341 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:37.535+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:37.884+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fedb8016bd0 con 0x7fedc4109a80 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:37.884+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 msgr2=0x7fedac040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:37.884+0000 7fedc2000700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.885+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fedb8055b30 con 0x7fedc4109a80 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.885+0000 7fedc2000700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 0x7fedac044000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.885+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fedb8009530 con 0x7fedb8053aa0 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.886+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 0x7fedac044000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.887+0000 7fedc8e00700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 0x7fedac044000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fedb0003a10 tx=0x7fedb0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.889+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fedb8009530 con 0x7fedb8053aa0 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.904+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fedc4064e10 con 0x7fedb8053aa0 2024-05-27T06:44:39.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.905+0000 7fedc2000700 1 -- 172.21.15.16:0/3350039349 <== mgr.14152 v2:172.21.15.16:6800/2755801148 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fedc4064e10 con 0x7fedb8053aa0 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.905+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 msgr2=0x7fedac044000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.905+0000 7fedcc200700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 0x7fedac044000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fedb0003a10 tx=0x7fedb0009270 comp rx=0 tx=0).stop 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.905+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 msgr2=0x7fedc41a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.905+0000 7fedcc200700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedb8006af0 tx=0x7fedb8011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 shutdown_connections 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fedb8053aa0 0x7fedac044000 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc4109a80 0x7fedc41a00c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 --2- 172.21.15.16:0/3350039349 >> [v2:172.21.15.16:6800/2390178341,v1:172.21.15.16:6801/2390178341] conn(0x7fedac03e520 0x7fedac040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 >> 172.21.15.16:0/3350039349 conn(0x7fedc41050c0 msgr2=0x7fedc407d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 shutdown_connections 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:38.906+0000 7fedcc200700 1 -- 172.21.15.16:0/3350039349 wait complete. 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-05-27T06:44:39.608 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-05-27T06:44:40.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:39 smithi016 ceph-mon[31415]: [27/May/2024:06:44:39] ENGINE Bus STARTING 2024-05-27T06:44:40.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:39 smithi016 ceph-mon[31415]: [27/May/2024:06:44:39] ENGINE Serving on https://172.21.15.16:7150 2024-05-27T06:44:40.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:39 smithi016 ceph-mon[31415]: [27/May/2024:06:44:39] ENGINE Serving on http://172.21.15.16:8765 2024-05-27T06:44:40.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:39 smithi016 ceph-mon[31415]: [27/May/2024:06:44:39] ENGINE Bus STARTED 2024-05-27T06:44:41.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:40 smithi016 ceph-mon[31415]: mgrmap e12: x(active, since 2s) 2024-05-27T06:44:41.896 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T06:44:41.896 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.579+0000 7fcca6800700 1 Processor -- start 2024-05-27T06:44:41.896 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.579+0000 7fcca6800700 1 -- start start 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.580+0000 7fcca6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.580+0000 7fcca6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcca007e100 con 0x7fcca007f700 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.580+0000 7fcc9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.580+0000 7fcc9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55706/0 (socket says 172.21.15.16:55706) 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.580+0000 7fcc9f400700 1 -- 172.21.15.16:0/670894030 learned_addr learned my addr 172.21.15.16:0/670894030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.581+0000 7fcc9f400700 1 -- 172.21.15.16:0/670894030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca007e280 con 0x7fcca007f700 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.581+0000 7fcc9f400700 1 --2- 172.21.15.16:0/670894030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc900098a0 tx=0x7fcc90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1da029a585c41b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.582+0000 7fcc9ea00700 1 -- 172.21.15.16:0/670894030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90004160 con 0x7fcca007f700 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.582+0000 7fcc9ea00700 1 -- 172.21.15.16:0/670894030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc90004300 con 0x7fcca007f700 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.582+0000 7fcc9ea00700 1 -- 172.21.15.16:0/670894030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90011960 con 0x7fcca007f700 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 -- 172.21.15.16:0/670894030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 msgr2=0x7fcca007db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 --2- 172.21.15.16:0/670894030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc900098a0 tx=0x7fcc90009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:41.897 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 -- 172.21.15.16:0/670894030 shutdown_connections 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 --2- 172.21.15.16:0/670894030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca007db30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 -- 172.21.15.16:0/670894030 >> 172.21.15.16:0/670894030 conn(0x7fcca0105090 msgr2=0x7fcca0107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 -- 172.21.15.16:0/670894030 shutdown_connections 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.583+0000 7fcca6800700 1 -- 172.21.15.16:0/670894030 wait complete. 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.584+0000 7fcca6800700 1 Processor -- start 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.584+0000 7fcca6800700 1 -- start start 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.585+0000 7fcca6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.585+0000 7fcca6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0110050 con 0x7fcca007f700 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.585+0000 7fcc9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.585+0000 7fcc9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55714/0 (socket says 172.21.15.16:55714) 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.585+0000 7fcc9f400700 1 -- 172.21.15.16:0/4120694017 learned_addr learned my addr 172.21.15.16:0/4120694017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcc9f400700 1 -- 172.21.15.16:0/4120694017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca01a04a0 con 0x7fcca007f700 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcc9f400700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009520 tx=0x7fcc90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90003aa0 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9001b8d0 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca01a0730 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90023790 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.586+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca01a3340 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.588+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca0064e10 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.588+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fcc9001b440 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.589+0000 7fcc9cc00700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 0x7fcc84040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.589+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc90054090 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.589+0000 7fcc96a00700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 0x7fcc84040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.590+0000 7fcc96a00700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 0x7fcc84040a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800ad30 tx=0x7fcc880093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.595+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc9001f030 con 0x7fcca007f700 2024-05-27T06:44:41.899 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:40.840+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fcca0106000 con 0x7fcc8403e520 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.093+0000 7fcc9cc00700 1 -- 172.21.15.16:0/4120694017 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fcca0106000 con 0x7fcc8403e520 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 msgr2=0x7fcc84040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 0x7fcc84040a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800ad30 tx=0x7fcc880093f0 comp rx=0 tx=0).stop 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 msgr2=0x7fcca019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009520 tx=0x7fcc90011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 shutdown_connections 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc8403e520 0x7fcc84040a30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 --2- 172.21.15.16:0/4120694017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcca007f700 0x7fcca019ff60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 >> 172.21.15.16:0/4120694017 conn(0x7fcca0105090 msgr2=0x7fcca01058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.097+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 shutdown_connections 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:41.098+0000 7fcca6800700 1 -- 172.21.15.16:0/4120694017 wait complete. 2024-05-27T06:44:41.900 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-05-27T06:44:42.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:42 smithi016 ceph-mon[31415]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:42.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:42.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:42.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$XiBuUBDILSOm1uLy.tDWTu55R/jlFdJlVQqAC7Q4TyTk4yN.wRNMS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716792283, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.570+0000 7f4787a00700 1 Processor -- start 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.570+0000 7f4787a00700 1 -- start start 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.571+0000 7f4787a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.571+0000 7f4787a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f478007d8b0 con 0x7f478010a4f0 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.571+0000 7f4785000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.571+0000 7f4785000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58040/0 (socket says 172.21.15.16:58040) 2024-05-27T06:44:43.792 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.571+0000 7f4785000700 1 -- 172.21.15.16:0/1737631306 learned_addr learned my addr 172.21.15.16:0/1737631306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.572+0000 7f4785000700 1 -- 172.21.15.16:0/1737631306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478010ce50 con 0x7f478010a4f0 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.572+0000 7f4785000700 1 --2- 172.21.15.16:0/1737631306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47740098a0 tx=0x7f4774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5696dcf2baad96a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.573+0000 7f477f400700 1 -- 172.21.15.16:0/1737631306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774004160 con 0x7f478010a4f0 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.573+0000 7f477f400700 1 -- 172.21.15.16:0/1737631306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4774004300 con 0x7f478010a4f0 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.573+0000 7f477f400700 1 -- 172.21.15.16:0/1737631306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774011960 con 0x7f478010a4f0 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 -- 172.21.15.16:0/1737631306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 msgr2=0x7f478010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 --2- 172.21.15.16:0/1737631306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47740098a0 tx=0x7f4774009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 -- 172.21.15.16:0/1737631306 shutdown_connections 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 --2- 172.21.15.16:0/1737631306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478010c910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 -- 172.21.15.16:0/1737631306 >> 172.21.15.16:0/1737631306 conn(0x7f4780103f50 msgr2=0x7f4780106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 -- 172.21.15.16:0/1737631306 shutdown_connections 2024-05-27T06:44:43.793 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.574+0000 7f4787a00700 1 -- 172.21.15.16:0/1737631306 wait complete. 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.575+0000 7f4787a00700 1 Processor -- start 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.575+0000 7f4787a00700 1 -- start start 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.576+0000 7f4787a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.576+0000 7f4787a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f478010ddf0 con 0x7f478010a4f0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.576+0000 7f4785000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.576+0000 7f4785000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58056/0 (socket says 172.21.15.16:58056) 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.576+0000 7f4785000700 1 -- 172.21.15.16:0/3502070404 learned_addr learned my addr 172.21.15.16:0/3502070404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.577+0000 7f4785000700 1 -- 172.21.15.16:0/3502070404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47801a04a0 con 0x7f478010a4f0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.577+0000 7f4785000700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4774011670 tx=0x7f4774011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.577+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f477401f070 con 0x7f478010a4f0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.577+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4774003b20 con 0x7f478010a4f0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.578+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47801a0730 con 0x7f478010a4f0 2024-05-27T06:44:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.578+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774019580 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.578+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47801a3340 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.579+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4780064e10 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.579+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f4774022070 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.580+0000 7f477d600700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 0x7f4768040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.580+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4774053bf0 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.581+0000 7f477fe00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 0x7f4768040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.581+0000 7f477fe00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 0x7f4768040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.586+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f477402cd00 con 0x7f478010a4f0 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:42.842+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f47801a35d0 con 0x7f476803e520 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.120+0000 7f477d600700 1 -- 172.21.15.16:0/3502070404 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f47801a35d0 con 0x7f476803e520 2024-05-27T06:44:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 msgr2=0x7f4768040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 0x7f4768040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 msgr2=0x7f478019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4774011670 tx=0x7f4774011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 shutdown_connections 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f476803e520 0x7f4768040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 --2- 172.21.15.16:0/3502070404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478010a4f0 0x7f478019ff60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 >> 172.21.15.16:0/3502070404 conn(0x7f4780103f50 msgr2=0x7f4780108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 shutdown_connections 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:43.123+0000 7f4787a00700 1 -- 172.21.15.16:0/3502070404 wait complete. 2024-05-27T06:44:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-05-27T06:44:44.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:44 smithi016 ceph-mon[31415]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:44:44.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:44.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:45.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:45 smithi016 ceph-mon[31415]: mgrmap e13: x(active, since 6s) 2024-05-27T06:44:45.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:45 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3470432145' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.550+0000 7fcc87e00700 1 Processor -- start 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.550+0000 7fcc87e00700 1 -- start start 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.550+0000 7fcc87e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.550+0000 7fcc87e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010a450 con 0x7fcc80109a50 2024-05-27T06:44:46.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.551+0000 7fcc85400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.551+0000 7fcc85400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58066/0 (socket says 172.21.15.16:58066) 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.551+0000 7fcc85400700 1 -- 172.21.15.16:0/1573940904 learned_addr learned my addr 172.21.15.16:0/1573940904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.551+0000 7fcc85400700 1 -- 172.21.15.16:0/1573940904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8010ac20 con 0x7fcc80109a50 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.552+0000 7fcc85400700 1 --2- 172.21.15.16:0/1573940904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc740098a0 tx=0x7fcc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0735a07e86c8e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.552+0000 7fcc7fe00700 1 -- 172.21.15.16:0/1573940904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74004160 con 0x7fcc80109a50 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.552+0000 7fcc7fe00700 1 -- 172.21.15.16:0/1573940904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc74004300 con 0x7fcc80109a50 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.552+0000 7fcc7fe00700 1 -- 172.21.15.16:0/1573940904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74011960 con 0x7fcc80109a50 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.553+0000 7fcc87e00700 1 -- 172.21.15.16:0/1573940904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 msgr2=0x7fcc80109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.553+0000 7fcc87e00700 1 --2- 172.21.15.16:0/1573940904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc740098a0 tx=0x7fcc74009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.553+0000 7fcc87e00700 1 -- 172.21.15.16:0/1573940904 shutdown_connections 2024-05-27T06:44:46.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.553+0000 7fcc87e00700 1 --2- 172.21.15.16:0/1573940904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc80109e60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.553+0000 7fcc87e00700 1 -- 172.21.15.16:0/1573940904 >> 172.21.15.16:0/1573940904 conn(0x7fcc801050b0 msgr2=0x7fcc80107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.554+0000 7fcc87e00700 1 -- 172.21.15.16:0/1573940904 shutdown_connections 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.554+0000 7fcc87e00700 1 -- 172.21.15.16:0/1573940904 wait complete. 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.555+0000 7fcc87e00700 1 Processor -- start 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.555+0000 7fcc87e00700 1 -- start start 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.555+0000 7fcc87e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.555+0000 7fcc87e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010dfd0 con 0x7fcc80109a50 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.556+0000 7fcc85400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.556+0000 7fcc85400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58076/0 (socket says 172.21.15.16:58076) 2024-05-27T06:44:46.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.556+0000 7fcc85400700 1 -- 172.21.15.16:0/3470432145 learned_addr learned my addr 172.21.15.16:0/3470432145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.556+0000 7fcc85400700 1 -- 172.21.15.16:0/3470432145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8019f520 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.556+0000 7fcc85400700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc740119b0 tx=0x7fcc74011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.557+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74003aa0 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.557+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc7401b8d0 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.557+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc740237d0 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.557+0000 7fcc87e00700 1 -- 172.21.15.16:0/3470432145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8019f7b0 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.557+0000 7fcc87e00700 1 -- 172.21.15.16:0/3470432145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8019bc00 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.558+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c005330 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.559+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcc7401b440 con 0x7fcc80109a50 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.559+0000 7fcc7e000700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 0x7fcc68040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:46.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.559+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc74054170 con 0x7fcc80109a50 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.560+0000 7fcc84a00700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 0x7fcc68040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.561+0000 7fcc84a00700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 0x7fcc68040a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006fd0 tx=0x7fcc6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.565+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc7401f030 con 0x7fcc80109a50 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.802+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcc5c005620 con 0x7fcc80109a50 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.803+0000 7fcc7e000700 1 -- 172.21.15.16:0/3470432145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcc7402b3a0 con 0x7fcc80109a50 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 msgr2=0x7fcc68040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 0x7fcc68040a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006fd0 tx=0x7fcc6c006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 msgr2=0x7fcc8019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc740119b0 tx=0x7fcc74011a90 comp rx=0 tx=0).stop 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 shutdown_connections 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcc6803e570 0x7fcc68040a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:46.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 --2- 172.21.15.16:0/3470432145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80109a50 0x7fcc8019efe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:46.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.806+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 >> 172.21.15.16:0/3470432145 conn(0x7fcc801050b0 msgr2=0x7fcc8007d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:46.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.807+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 shutdown_connections 2024-05-27T06:44:46.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:44.807+0000 7fcc73400700 1 -- 172.21.15.16:0/3470432145 wait complete. 2024-05-27T06:44:46.595 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T06:44:46.595 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-27T06:44:46.613 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T06:44:46.613 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-27T06:44:46.613 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-27T06:44:46.613 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout: Password: 2vja2rggg4 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:46.617 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config directory 2024-05-27T06:44:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling autotune for osd_memory_target 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.292+0000 7feea0800700 1 Processor -- start 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.292+0000 7feea0800700 1 -- start start 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.293+0000 7feea0800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.293+0000 7feea0800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee9807d870 con 0x7fee9810a530 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.293+0000 7fee9de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.293+0000 7fee9de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58090/0 (socket says 172.21.15.16:58090) 2024-05-27T06:44:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.293+0000 7fee9de00700 1 -- 172.21.15.16:0/1765036430 learned_addr learned my addr 172.21.15.16:0/1765036430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.294+0000 7fee9de00700 1 -- 172.21.15.16:0/1765036430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9810ce90 con 0x7fee9810a530 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.294+0000 7fee9de00700 1 --2- 172.21.15.16:0/1765036430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098a0 tx=0x7fee8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2371f07b8d3dae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.294+0000 7fee9ca00700 1 -- 172.21.15.16:0/1765036430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee8c004160 con 0x7fee9810a530 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.294+0000 7fee9ca00700 1 -- 172.21.15.16:0/1765036430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee8c004300 con 0x7fee9810a530 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.295+0000 7fee9ca00700 1 -- 172.21.15.16:0/1765036430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee8c011960 con 0x7fee9810a530 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.295+0000 7feea0800700 1 -- 172.21.15.16:0/1765036430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 msgr2=0x7fee9810c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.295+0000 7feea0800700 1 --2- 172.21.15.16:0/1765036430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098a0 tx=0x7fee8c009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.296+0000 7feea0800700 1 -- 172.21.15.16:0/1765036430 shutdown_connections 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.296+0000 7feea0800700 1 --2- 172.21.15.16:0/1765036430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9810c950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.296+0000 7feea0800700 1 -- 172.21.15.16:0/1765036430 >> 172.21.15.16:0/1765036430 conn(0x7fee98103f50 msgr2=0x7fee98106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.296+0000 7feea0800700 1 -- 172.21.15.16:0/1765036430 shutdown_connections 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.296+0000 7feea0800700 1 -- 172.21.15.16:0/1765036430 wait complete. 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.297+0000 7feea0800700 1 Processor -- start 2024-05-27T06:44:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7feea0800700 1 -- start start 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7feea0800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7feea0800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810ddd0 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7fee9de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7fee9de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58106/0 (socket says 172.21.15.16:58106) 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.298+0000 7fee9de00700 1 -- 172.21.15.16:0/3431882031 learned_addr learned my addr 172.21.15.16:0/3431882031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.299+0000 7fee9de00700 1 -- 172.21.15.16:0/3431882031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9819e260 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.299+0000 7fee9de00700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee8c011670 tx=0x7fee8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.299+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee8c01f070 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.300+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9819e4f0 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.300+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee8c003b20 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.300+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee8c019580 con 0x7fee9810a530 2024-05-27T06:44:48.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.300+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a10f0 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.301+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98064e10 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.301+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fee8c022070 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.302+0000 7fee92a00700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 0x7fee80040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.302+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee8c053d80 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.302+0000 7fee9d400700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 0x7fee80040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.303+0000 7fee9d400700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 0x7fee80040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee88006fd0 tx=0x7fee88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.308+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee8c019840 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.547+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fee981a1400 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.548+0000 7fee92a00700 1 -- 172.21.15.16:0/3431882031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fee8c00bd00 con 0x7fee9810a530 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.552+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 msgr2=0x7fee80040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.552+0000 7feea0800700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 0x7fee80040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee88006fd0 tx=0x7fee88006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.552+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 msgr2=0x7fee9819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.552+0000 7feea0800700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee8c011670 tx=0x7fee8c011a30 comp rx=0 tx=0).stop 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 shutdown_connections 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fee8003e560 0x7fee80040a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 --2- 172.21.15.16:0/3431882031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9810a530 0x7fee9819dd20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 >> 172.21.15.16:0/3431882031 conn(0x7fee98103f50 msgr2=0x7fee981047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 shutdown_connections 2024-05-27T06:44:48.408 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:47.553+0000 7feea0800700 1 -- 172.21.15.16:0/3431882031 wait complete. 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.097+0000 7fa1c2800700 1 Processor -- start 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.098+0000 7fa1c2800700 1 -- start start 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.098+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.098+0000 7fa1c2800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10a450 con 0x7fa1bc109a50 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.099+0000 7fa1bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.099+0000 7fa1bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.099+0000 7fa1bb400700 1 -- 172.21.15.16:0/1467938477 learned_addr learned my addr 172.21.15.16:0/1467938477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.100+0000 7fa1bb400700 1 -- 172.21.15.16:0/1467938477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc10ac20 con 0x7fa1bc109a50 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.100+0000 7fa1bb400700 1 --2- 172.21.15.16:0/1467938477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae5cabebd2e7095 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:50.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.100+0000 7fa1ba000700 1 -- 172.21.15.16:0/1467938477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac004160 con 0x7fa1bc109a50 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.100+0000 7fa1ba000700 1 -- 172.21.15.16:0/1467938477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1ac004300 con 0x7fa1bc109a50 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.100+0000 7fa1ba000700 1 -- 172.21.15.16:0/1467938477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac011960 con 0x7fa1bc109a50 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.101+0000 7fa1c2800700 1 -- 172.21.15.16:0/1467938477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 msgr2=0x7fa1bc109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.101+0000 7fa1c2800700 1 --2- 172.21.15.16:0/1467938477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).stop 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.101+0000 7fa1c2800700 1 -- 172.21.15.16:0/1467938477 shutdown_connections 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.101+0000 7fa1c2800700 1 --2- 172.21.15.16:0/1467938477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc109e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.101+0000 7fa1c2800700 1 -- 172.21.15.16:0/1467938477 >> 172.21.15.16:0/1467938477 conn(0x7fa1bc1050b0 msgr2=0x7fa1bc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.102+0000 7fa1c2800700 1 -- 172.21.15.16:0/1467938477 shutdown_connections 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.102+0000 7fa1c2800700 1 -- 172.21.15.16:0/1467938477 wait complete. 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.103+0000 7fa1c2800700 1 Processor -- start 2024-05-27T06:44:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.103+0000 7fa1c2800700 1 -- start start 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.103+0000 7fa1c2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.103+0000 7fa1c2800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10dfd0 con 0x7fa1bc109a50 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.104+0000 7fa1bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.104+0000 7fa1bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58126/0 (socket says 172.21.15.16:58126) 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.104+0000 7fa1bb400700 1 -- 172.21.15.16:0/162400172 learned_addr learned my addr 172.21.15.16:0/162400172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.104+0000 7fa1bb400700 1 -- 172.21.15.16:0/162400172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc1a0530 con 0x7fa1bc109a50 2024-05-27T06:44:50.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.104+0000 7fa1bb400700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac011980 tx=0x7fa1ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.105+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac01f070 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.105+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1ac003aa0 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.105+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc1a07c0 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.105+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac019560 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.105+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a33d0 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.106+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc064e10 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.106+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa1ac022070 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.107+0000 7fa1b3e00700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 0x7fa1a0040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.107+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1ac053c70 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.107+0000 7fa1baa00700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 0x7fa1a0040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.108+0000 7fa1baa00700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 0x7fa1a0040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.113+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1ac023aa0 con 0x7fa1bc109a50 2024-05-27T06:44:50.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.472+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa1bc0643c0 con 0x7fa1bc109a50 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.478+0000 7fa1b3e00700 1 -- 172.21.15.16:0/162400172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fa1ac01ba80 con 0x7fa1bc109a50 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.481+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 msgr2=0x7fa1a0040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.481+0000 7fa1c2800700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 0x7fa1a0040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 msgr2=0x7fa1bc19fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac011980 tx=0x7fa1ac011a60 comp rx=0 tx=0).stop 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 shutdown_connections 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1a003e560 0x7fa1a0040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 --2- 172.21.15.16:0/162400172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc109a50 0x7fa1bc19fff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 >> 172.21.15.16:0/162400172 conn(0x7fa1bc1050b0 msgr2=0x7fa1bc07d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 shutdown_connections 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-27T06:44:49.482+0000 7fa1c2800700 1 -- 172.21.15.16:0/162400172 wait complete. 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T06:44:50.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T06:44:50.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:44:50.407 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-05-27T06:44:50.512 INFO:tasks.cephadm:Fetching config... 2024-05-27T06:44:50.512 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:44:50.513 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T06:44:50.568 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T06:44:50.568 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:44:50.568 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T06:44:50.615 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T06:44:50.615 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:44:50.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-27T06:44:50.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:50 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/162400172' entity='client.admin' 2024-05-27T06:44:50.728 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T06:44:50.729 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:44:50.729 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T06:44:50.825 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T06:44:50.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T06:44:50.957 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:44:50.994 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T06:44:51.112 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:44:51.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T06:44:51.287 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCQcHllyWDy2EmP9mFK0HaMUzApkvK0fopcidC2brA0wFPsWuDwLTK2j7PEJkn7W6a//Iy5avl0jD5ndIN89qjBwCPHIgqtd98V0NgTPLU1QIawFvr7qyiklb69ozqnYFnNgpQzTliKbo8vxmhudwDfxIlOJsDUBARTi6TO3kwgXlcn/jllRN53zu8LcRrg5YYcsVq9nUZeHxHLEtsOi9STh/izDz+P9ahB430OSz3nUVs+sQ+0wjUH2X2F/7nnL9Ro5XWDmq1W+kf2zBjbbRyHgz15N8BXaOPtuawROSRPt+JcmGanslsDnOSs6yGbfK28vQUhysWb/xrN8Jcefhsc4JiSIx3TXjWpKAf0ike7dYgUTt3A0upONAjso476G9bsbPpYjjsVYpiZxy30dDFsyIDHJAI2J6ULCzZdqWJqQ9PWA/Gstax330sD0fwx8+Jp63w1lKj5LD2fbLg27cwFuSxn5ptWlA97BTuGPVHVEtACGFYOVs2WnbaQYczD3ms= ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:44:51.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T06:44:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.769+0000 7f7da6000700 1 -- 172.21.15.16:0/3167577739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 msgr2=0x7f7da0106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.769+0000 7f7da6000700 1 --2- 172.21.15.16:0/3167577739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da0106050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009910 tx=0x7f7d94009c20 comp rx=0 tx=0).stop 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.769+0000 7f7da6000700 1 -- 172.21.15.16:0/3167577739 shutdown_connections 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.769+0000 7f7da6000700 1 --2- 172.21.15.16:0/3167577739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da0106050 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.769+0000 7f7da6000700 1 -- 172.21.15.16:0/3167577739 >> 172.21.15.16:0/3167577739 conn(0x7f7da0101260 msgr2=0x7f7da0103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.770+0000 7f7da6000700 1 -- 172.21.15.16:0/3167577739 shutdown_connections 2024-05-27T06:44:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.770+0000 7f7da6000700 1 -- 172.21.15.16:0/3167577739 wait complete. 2024-05-27T06:44:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.771+0000 7f7da6000700 1 Processor -- start 2024-05-27T06:44:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da6000700 1 -- start start 2024-05-27T06:44:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da6000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da6000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010bab0 con 0x7f7da0105c40 2024-05-27T06:44:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da4c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da4c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41010/0 (socket says 172.21.15.16:41010) 2024-05-27T06:44:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.772+0000 7f7da4c00700 1 -- 172.21.15.16:0/1496825062 learned_addr learned my addr 172.21.15.16:0/1496825062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7da4c00700 1 -- 172.21.15.16:0/1496825062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da019bbe0 con 0x7f7da0105c40 2024-05-27T06:44:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7da4c00700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005760 tx=0x7f7d94004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d94027070 con 0x7f7da0105c40 2024-05-27T06:44:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d9400bdf0 con 0x7f7da0105c40 2024-05-27T06:44:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019be70 con 0x7f7da0105c40 2024-05-27T06:44:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.773+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d9400fd70 con 0x7f7da0105c40 2024-05-27T06:44:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.774+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da019ea80 con 0x7f7da0105c40 2024-05-27T06:44:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.775+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da010c230 con 0x7f7da0105c40 2024-05-27T06:44:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.775+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7d9402a070 con 0x7f7da0105c40 2024-05-27T06:44:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.775+0000 7f7d9d600700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 0x7f7d88040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.775+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d94053940 con 0x7f7da0105c40 2024-05-27T06:44:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.776+0000 7f7d9fe00700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 0x7f7d88040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:54.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.777+0000 7f7d9fe00700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 0x7f7d88040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:54.781+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d94013070 con 0x7f7da0105c40 2024-05-27T06:44:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.016+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f7da002e6f0 con 0x7f7da0105c40 2024-05-27T06:44:55.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.023+0000 7f7d9d600700 1 -- 172.21.15.16:0/1496825062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f7d9400f6c0 con 0x7f7da0105c40 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:55.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.026+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 msgr2=0x7f7d88040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.026+0000 7f7da6000700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 0x7f7d88040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.026+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 msgr2=0x7f7da019b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.026+0000 7f7da6000700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005760 tx=0x7f7d94004f60 comp rx=0 tx=0).stop 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.027+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 shutdown_connections 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.027+0000 7f7da6000700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7d8803e570 0x7f7d88040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.027+0000 7f7da6000700 1 --2- 172.21.15.16:0/1496825062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da0105c40 0x7f7da019b6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.027+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 >> 172.21.15.16:0/1496825062 conn(0x7f7da0101260 msgr2=0x7f7da0102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.027+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 shutdown_connections 2024-05-27T06:44:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:55.028+0000 7f7da6000700 1 -- 172.21.15.16:0/1496825062 wait complete. 2024-05-27T06:44:55.835 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T06:44:55.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T06:44:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:56 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1496825062' entity='client.admin' 2024-05-27T06:44:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:44:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 -- 172.21.15.16:0/2930923363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 msgr2=0x7fbb6c106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 --2- 172.21.15.16:0/2930923363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c106050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009910 tx=0x7fbb5c009c20 comp rx=0 tx=0).stop 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 -- 172.21.15.16:0/2930923363 shutdown_connections 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 --2- 172.21.15.16:0/2930923363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c106050 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 -- 172.21.15.16:0/2930923363 >> 172.21.15.16:0/2930923363 conn(0x7fbb6c101260 msgr2=0x7fbb6c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 -- 172.21.15.16:0/2930923363 shutdown_connections 2024-05-27T06:44:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.358+0000 7fbb6be00700 1 -- 172.21.15.16:0/2930923363 wait complete. 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.359+0000 7fbb6be00700 1 Processor -- start 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6be00700 1 -- start start 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6be00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c10bab0 con 0x7fbb6c105c40 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41028/0 (socket says 172.21.15.16:41028) 2024-05-27T06:44:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.360+0000 7fbb6aa00700 1 -- 172.21.15.16:0/3162195680 learned_addr learned my addr 172.21.15.16:0/3162195680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:44:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.361+0000 7fbb6aa00700 1 -- 172.21.15.16:0/3162195680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c19bbe0 con 0x7fbb6c105c40 2024-05-27T06:44:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.361+0000 7fbb6aa00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005950 tx=0x7fbb5c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.361+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c027070 con 0x7fbb6c105c40 2024-05-27T06:44:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.361+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb5c00bdf0 con 0x7fbb6c105c40 2024-05-27T06:44:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.361+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c19be70 con 0x7fbb6c105c40 2024-05-27T06:44:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.362+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c00fdb0 con 0x7fbb6c105c40 2024-05-27T06:44:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.362+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c19ea80 con 0x7fbb6c105c40 2024-05-27T06:44:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.363+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6c10c230 con 0x7fbb6c105c40 2024-05-27T06:44:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.363+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbb5c02a070 con 0x7fbb6c105c40 2024-05-27T06:44:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.364+0000 7fbb70a00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 0x7fbb50040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:44:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.364+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb5c0539f0 con 0x7fbb6c105c40 2024-05-27T06:44:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.364+0000 7fbb6a000700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 0x7fbb50040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:44:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.365+0000 7fbb6a000700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 0x7fbb50040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb58006fd0 tx=0x7fbb58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:44:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.370+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb5c013070 con 0x7fbb6c105c40 2024-05-27T06:44:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.612+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fbb6c02e6f0 con 0x7fbb5003e510 2024-05-27T06:44:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.620+0000 7fbb70a00700 1 -- 172.21.15.16:0/3162195680 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbb6c02e6f0 con 0x7fbb5003e510 2024-05-27T06:44:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.623+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 msgr2=0x7fbb50040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.623+0000 7fbb6be00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 0x7fbb50040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb58006fd0 tx=0x7fbb58006e40 comp rx=0 tx=0).stop 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.623+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 msgr2=0x7fbb6c19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.623+0000 7fbb6be00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005950 tx=0x7fbb5c005810 comp rx=0 tx=0).stop 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 shutdown_connections 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbb5003e510 0x7fbb50040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 --2- 172.21.15.16:0/3162195680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c105c40 0x7fbb6c19b6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 >> 172.21.15.16:0/3162195680 conn(0x7fbb6c101260 msgr2=0x7fbb6c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 shutdown_connections 2024-05-27T06:44:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:44:58.624+0000 7fbb6be00700 1 -- 172.21.15.16:0/3162195680 wait complete. 2024-05-27T06:44:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:58 smithi016 ceph-mon[31415]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:44:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:44:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:44:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:44:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:44:59.496 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi069 2024-05-27T06:44:59.496 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:44:59.496 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.conf 2024-05-27T06:44:59.552 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:44:59.553 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:44:59.651 INFO:tasks.cephadm:Adding host smithi069 to orchestrator... 2024-05-27T06:44:59.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi069 2024-05-27T06:45:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:45:00.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:59 smithi016 ceph-mon[31415]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:00.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:44:59 smithi016 ceph-mon[31415]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T06:45:01.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:00 smithi016 ceph-mon[31415]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:01.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:00 smithi016 ceph-mon[31415]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:01.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:00 smithi016 ceph-mon[31415]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:02.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 -- 172.21.15.16:0/2647444521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 msgr2=0x7f095c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:02.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 --2- 172.21.15.16:0/2647444521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c076130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f094c009910 tx=0x7f094c009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 -- 172.21.15.16:0/2647444521 shutdown_connections 2024-05-27T06:45:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 --2- 172.21.15.16:0/2647444521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c076130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 -- 172.21.15.16:0/2647444521 >> 172.21.15.16:0/2647444521 conn(0x7f095c0ff480 msgr2=0x7f095c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 -- 172.21.15.16:0/2647444521 shutdown_connections 2024-05-27T06:45:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.331+0000 7f0961c00700 1 -- 172.21.15.16:0/2647444521 wait complete. 2024-05-27T06:45:02.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:02 smithi016 ceph-mon[31415]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:02.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:02.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:02.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.333+0000 7f0961c00700 1 Processor -- start 2024-05-27T06:45:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.333+0000 7f0961c00700 1 -- start start 2024-05-27T06:45:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f0961c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f0961c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10ffd0 con 0x7f095c075d20 2024-05-27T06:45:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f095be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f095be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40906/0 (socket says 172.21.15.16:40906) 2024-05-27T06:45:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f095be00700 1 -- 172.21.15.16:0/1658866198 learned_addr learned my addr 172.21.15.16:0/1658866198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:45:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.334+0000 7f095be00700 1 -- 172.21.15.16:0/1658866198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a1ae0 con 0x7f095c075d20 2024-05-27T06:45:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f095be00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f094c005060 tx=0x7f094c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c026070 con 0x7f095c075d20 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f094c00b750 con 0x7f095c075d20 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c003850 con 0x7f095c075d20 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a1d70 con 0x7f095c075d20 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.335+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c19b730 con 0x7f095c075d20 2024-05-27T06:45:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.336+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c077290 con 0x7f095c075d20 2024-05-27T06:45:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.337+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f094c029070 con 0x7f095c075d20 2024-05-27T06:45:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.337+0000 7f0958c00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 0x7f0940040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.337+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f094c01e070 con 0x7f095c075d20 2024-05-27T06:45:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.337+0000 7f095b400700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 0x7f0940040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.338+0000 7f095b400700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 0x7f0940040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0948006fd0 tx=0x7f0948006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.343+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f094c013070 con 0x7f095c075d20 2024-05-27T06:45:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:02.575+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}) -- 0x7f095c19cc40 con 0x7f094003e570 2024-05-27T06:45:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:03 smithi016 ceph-mon[31415]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:04.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:04 smithi016 ceph-mon[31415]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:05 smithi016 ceph-mon[31415]: Deploying cephadm binary to smithi069 2024-05-27T06:45:05.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:05 smithi016 ceph-mon[31415]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:07.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:07 smithi016 ceph-mon[31415]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.827+0000 7f0958c00700 1 -- 172.21.15.16:0/1658866198 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f095c19cc40 con 0x7f094003e570 2024-05-27T06:45:08.820 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi069' with addr '172.21.15.69' 2024-05-27T06:45:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 msgr2=0x7f0940040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 0x7f0940040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0948006fd0 tx=0x7f0948006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 msgr2=0x7f095c1a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f094c005060 tx=0x7f094c005710 comp rx=0 tx=0).stop 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 shutdown_connections 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f094003e570 0x7f0940040a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 --2- 172.21.15.16:0/1658866198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c075d20 0x7f095c1a15a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 >> 172.21.15.16:0/1658866198 conn(0x7f095c0ff480 msgr2=0x7f095c10c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 shutdown_connections 2024-05-27T06:45:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:08.831+0000 7f0961c00700 1 -- 172.21.15.16:0/1658866198 wait complete. 2024-05-27T06:45:09.533 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:09 smithi016 ceph-mon[31415]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:09.533 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:09.533 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:09.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T06:45:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:45:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:10 smithi016 ceph-mon[31415]: Added host smithi069 2024-05-27T06:45:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:11.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:11 smithi016 ceph-mon[31415]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:11.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:11 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.305+0000 7f6d14c00700 1 -- 172.21.15.16:0/3806939861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 msgr2=0x7f6d1010a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.305+0000 7f6d14c00700 1 --2- 172.21.15.16:0/3806939861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1010a7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009910 tx=0x7f6d00009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.305+0000 7f6d14c00700 1 -- 172.21.15.16:0/3806939861 shutdown_connections 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.305+0000 7f6d14c00700 1 --2- 172.21.15.16:0/3806939861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1010a7d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.305+0000 7f6d14c00700 1 -- 172.21.15.16:0/3806939861 >> 172.21.15.16:0/3806939861 conn(0x7f6d100799e0 msgr2=0x7f6d101085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.306+0000 7f6d14c00700 1 -- 172.21.15.16:0/3806939861 shutdown_connections 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.306+0000 7f6d14c00700 1 -- 172.21.15.16:0/3806939861 wait complete. 2024-05-27T06:45:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.307+0000 7f6d14c00700 1 Processor -- start 2024-05-27T06:45:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.307+0000 7f6d14c00700 1 -- start start 2024-05-27T06:45:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.308+0000 7f6d14c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.308+0000 7f6d14c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10114560 con 0x7f6d1010a3c0 2024-05-27T06:45:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.308+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.308+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39756/0 (socket says 172.21.15.16:39756) 2024-05-27T06:45:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.308+0000 7f6d0de00700 1 -- 172.21.15.16:0/340021522 learned_addr learned my addr 172.21.15.16:0/340021522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:45:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.309+0000 7f6d0de00700 1 -- 172.21.15.16:0/340021522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d101a0390 con 0x7f6d1010a3c0 2024-05-27T06:45:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.309+0000 7f6d0de00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005660 tx=0x7f6d00005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.309+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d00026070 con 0x7f6d1010a3c0 2024-05-27T06:45:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.309+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d0000b6f0 con 0x7f6d1010a3c0 2024-05-27T06:45:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.309+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101a0680 con 0x7f6d1010a3c0 2024-05-27T06:45:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.310+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d000037b0 con 0x7f6d1010a3c0 2024-05-27T06:45:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.310+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101a4680 con 0x7f6d1010a3c0 2024-05-27T06:45:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.311+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6d00029070 con 0x7f6d1010a3c0 2024-05-27T06:45:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.311+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d1010b870 con 0x7f6d1010a3c0 2024-05-27T06:45:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.312+0000 7f6d06a00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 0x7f6cf4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.312+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d0001e070 con 0x7f6d1010a3c0 2024-05-27T06:45:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.312+0000 7f6d0d400700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 0x7f6cf4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.313+0000 7f6d0d400700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 0x7f6cf4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006fd0 tx=0x7f6cfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.318+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d00013070 con 0x7f6d1010a3c0 2024-05-27T06:45:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.563+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d101a11b0 con 0x7f6cf403e570 2024-05-27T06:45:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.565+0000 7f6d06a00700 1 -- 172.21.15.16:0/340021522 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f6d101a11b0 con 0x7f6cf403e570 2024-05-27T06:45:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:45:12.558 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}] 2024-05-27T06:45:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 msgr2=0x7f6cf4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 0x7f6cf4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006fd0 tx=0x7f6cfc006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 msgr2=0x7f6d1019fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005660 tx=0x7f6d00005740 comp rx=0 tx=0).stop 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 shutdown_connections 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6cf403e570 0x7f6cf4040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 --2- 172.21.15.16:0/340021522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1010a3c0 0x7f6d1019fe50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 >> 172.21.15.16:0/340021522 conn(0x7f6d100799e0 msgr2=0x7f6d1010bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.569+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 shutdown_connections 2024-05-27T06:45:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:12.570+0000 7f6d14c00700 1 -- 172.21.15.16:0/340021522 wait complete. 2024-05-27T06:45:13.320 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-05-27T06:45:13.320 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:45:13.320 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-05-27T06:45:13.380 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:45:13.380 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:13.475 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-05-27T06:45:13.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi164 2024-05-27T06:45:13.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:13 smithi016 ceph-mon[31415]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:13.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:13 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:45:14.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:14 smithi016 ceph-mon[31415]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:15.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:15 smithi016 ceph-mon[31415]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 -- 172.21.15.16:0/1827828078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 msgr2=0x7fc120076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 --2- 172.21.15.16:0/1827828078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc120076130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc110009910 tx=0x7fc110009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 -- 172.21.15.16:0/1827828078 shutdown_connections 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 --2- 172.21.15.16:0/1827828078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc120076130 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 -- 172.21.15.16:0/1827828078 >> 172.21.15.16:0/1827828078 conn(0x7fc1200ff690 msgr2=0x7fc120101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.096+0000 7fc125a00700 1 -- 172.21.15.16:0/1827828078 shutdown_connections 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.097+0000 7fc125a00700 1 -- 172.21.15.16:0/1827828078 wait complete. 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.098+0000 7fc125a00700 1 Processor -- start 2024-05-27T06:45:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.098+0000 7fc125a00700 1 -- start start 2024-05-27T06:45:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc125a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc125a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1201101e0 con 0x7fc120075d20 2024-05-27T06:45:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc11ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc11ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39774/0 (socket says 172.21.15.16:39774) 2024-05-27T06:45:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc11ea00700 1 -- 172.21.15.16:0/511670426 learned_addr learned my addr 172.21.15.16:0/511670426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:45:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.099+0000 7fc11ea00700 1 -- 172.21.15.16:0/511670426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1201a1c10 con 0x7fc120075d20 2024-05-27T06:45:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc11ea00700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc110006010 tx=0x7fc110005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc110026070 con 0x7fc120075d20 2024-05-27T06:45:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc11000b750 con 0x7fc120075d20 2024-05-27T06:45:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1201a1ea0 con 0x7fc120075d20 2024-05-27T06:45:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1100038a0 con 0x7fc120075d20 2024-05-27T06:45:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.100+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12019b8f0 con 0x7fc120075d20 2024-05-27T06:45:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.102+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc120077290 con 0x7fc120075d20 2024-05-27T06:45:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.102+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc110029070 con 0x7fc120075d20 2024-05-27T06:45:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.102+0000 7fc117400700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 0x7fc104040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.102+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc11001e070 con 0x7fc120075d20 2024-05-27T06:45:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.102+0000 7fc11e000700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 0x7fc104040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.103+0000 7fc11e000700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 0x7fc104040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006fd0 tx=0x7fc10c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.108+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc110013070 con 0x7fc120075d20 2024-05-27T06:45:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:16.351+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7fc12019ce20 con 0x7fc10403e510 2024-05-27T06:45:17.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:17 smithi016 ceph-mon[31415]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:18.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:19 smithi016 ceph-mon[31415]: Deploying cephadm binary to smithi164 2024-05-27T06:45:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:19 smithi016 ceph-mon[31415]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:45:19.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:19 smithi016 ceph-mon[31415]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:20.539 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:20 smithi016 ceph-mon[31415]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:21 smithi016 ceph-mon[31415]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:22.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:22 smithi016 ceph-mon[31415]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:22 smithi016 ceph-mon[31415]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.827+0000 7fc117400700 1 -- 172.21.15.16:0/511670426 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc12019ce20 con 0x7fc10403e510 2024-05-27T06:45:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-05-27T06:45:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.830+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 msgr2=0x7fc104040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.830+0000 7fc125a00700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 0x7fc104040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006fd0 tx=0x7fc10c006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 msgr2=0x7fc1201a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc110006010 tx=0x7fc110005710 comp rx=0 tx=0).stop 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 shutdown_connections 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc10403e510 0x7fc104040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 --2- 172.21.15.16:0/511670426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc120075d20 0x7fc1201a16d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 >> 172.21.15.16:0/511670426 conn(0x7fc1200ff690 msgr2=0x7fc12010c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 shutdown_connections 2024-05-27T06:45:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:22.831+0000 7fc125a00700 1 -- 172.21.15.16:0/511670426 wait complete. 2024-05-27T06:45:23.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T06:45:24.062 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:45:24.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:24.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: Added host smithi164 2024-05-27T06:45:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:25.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:24 smithi016 ceph-mon[31415]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:25.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:25.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:25.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 -- 172.21.15.16:0/2068941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 msgr2=0x7f7a90106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 --2- 172.21.15.16:0/2068941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a90106230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009910 tx=0x7f7a80009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 -- 172.21.15.16:0/2068941333 shutdown_connections 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 --2- 172.21.15.16:0/2068941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a90106230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 -- 172.21.15.16:0/2068941333 >> 172.21.15.16:0/2068941333 conn(0x7f7a901016b0 msgr2=0x7f7a90103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.221+0000 7f7a8f200700 1 -- 172.21.15.16:0/2068941333 shutdown_connections 2024-05-27T06:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.222+0000 7f7a8f200700 1 -- 172.21.15.16:0/2068941333 wait complete. 2024-05-27T06:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.223+0000 7f7a8f200700 1 Processor -- start 2024-05-27T06:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.223+0000 7f7a8f200700 1 -- start start 2024-05-27T06:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.223+0000 7f7a8f200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.223+0000 7f7a8f200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010ffc0 con 0x7f7a90105e20 2024-05-27T06:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.224+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.224+0000 7f7a8de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59220/0 (socket says 172.21.15.16:59220) 2024-05-27T06:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.224+0000 7f7a8de00700 1 -- 172.21.15.16:0/504334985 learned_addr learned my addr 172.21.15.16:0/504334985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.224+0000 7f7a8de00700 1 -- 172.21.15.16:0/504334985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a901a1ac0 con 0x7f7a90105e20 2024-05-27T06:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.224+0000 7f7a8de00700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005660 tx=0x7f7a80005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.225+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a80026070 con 0x7f7a90105e20 2024-05-27T06:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.225+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a8000b6f0 con 0x7f7a90105e20 2024-05-27T06:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.225+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a901a1db0 con 0x7f7a90105e20 2024-05-27T06:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.225+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a800037b0 con 0x7f7a90105e20 2024-05-27T06:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.225+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9019b8e0 con 0x7f7a90105e20 2024-05-27T06:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.226+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a901072d0 con 0x7f7a90105e20 2024-05-27T06:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.226+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7a80029070 con 0x7f7a90105e20 2024-05-27T06:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.227+0000 7f7a86a00700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 0x7f7a74040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.227+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a8001e070 con 0x7f7a90105e20 2024-05-27T06:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.227+0000 7f7a8d400700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 0x7f7a74040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.228+0000 7f7a8d400700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 0x7f7a74040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.233+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a80013070 con 0x7f7a90105e20 2024-05-27T06:45:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.481+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a9019cc50 con 0x7f7a7403e570 2024-05-27T06:45:26.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.483+0000 7f7a86a00700 1 -- 172.21.15.16:0/504334985 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f7a9019cc50 con 0x7f7a7403e570 2024-05-27T06:45:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:45:26.477 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 msgr2=0x7f7a74040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 0x7f7a74040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 msgr2=0x7f7a901a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005660 tx=0x7f7a80005740 comp rx=0 tx=0).stop 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 shutdown_connections 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7a7403e570 0x7f7a74040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 --2- 172.21.15.16:0/504334985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a90105e20 0x7f7a901a1580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 >> 172.21.15.16:0/504334985 conn(0x7f7a901016b0 msgr2=0x7f7a9010b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 shutdown_connections 2024-05-27T06:45:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:26.487+0000 7f7a8f200700 1 -- 172.21.15.16:0/504334985 wait complete. 2024-05-27T06:45:27.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:26 smithi016 ceph-mon[31415]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:27.272 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T06:45:27.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T06:45:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:45:28.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:27 smithi016 ceph-mon[31415]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:28.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:29.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:28 smithi016 ceph-mon[31415]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 -- 172.21.15.16:0/2426766422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 msgr2=0x7fe75c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 --2- 172.21.15.16:0/2426766422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c108f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009910 tx=0x7fe74c009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 -- 172.21.15.16:0/2426766422 shutdown_connections 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 --2- 172.21.15.16:0/2426766422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c108f90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 -- 172.21.15.16:0/2426766422 >> 172.21.15.16:0/2426766422 conn(0x7fe75c0fde20 msgr2=0x7fe75c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 -- 172.21.15.16:0/2426766422 shutdown_connections 2024-05-27T06:45:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.817+0000 7fe763000700 1 -- 172.21.15.16:0/2426766422 wait complete. 2024-05-27T06:45:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.818+0000 7fe763000700 1 Processor -- start 2024-05-27T06:45:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.818+0000 7fe763000700 1 -- start start 2024-05-27T06:45:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.819+0000 7fe763000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.819+0000 7fe763000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c10e0b0 con 0x7fe75c108b80 2024-05-27T06:45:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.819+0000 7fe75be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.819+0000 7fe75be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59244/0 (socket says 172.21.15.16:59244) 2024-05-27T06:45:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.819+0000 7fe75be00700 1 -- 172.21.15.16:0/3455136836 learned_addr learned my addr 172.21.15.16:0/3455136836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:45:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe75be00700 1 -- 172.21.15.16:0/3455136836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c19fac0 con 0x7fe75c108b80 2024-05-27T06:45:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe75be00700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b5c0 tx=0x7fe74c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c026070 con 0x7fe75c108b80 2024-05-27T06:45:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe74c00b750 con 0x7fe75c108b80 2024-05-27T06:45:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c003810 con 0x7fe75c108b80 2024-05-27T06:45:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c19fd50 con 0x7fe75c108b80 2024-05-27T06:45:29.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.820+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c199760 con 0x7fe75c108b80 2024-05-27T06:45:29.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.821+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c105290 con 0x7fe75c108b80 2024-05-27T06:45:29.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.822+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe74c029070 con 0x7fe75c108b80 2024-05-27T06:45:29.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.822+0000 7fe759600700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 0x7fe73c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:29.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.823+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe74c01e070 con 0x7fe75c108b80 2024-05-27T06:45:29.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.823+0000 7fe753400700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 0x7fe73c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:29.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.824+0000 7fe753400700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 0x7fe73c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:29.828+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe74c013070 con 0x7fe75c108b80 2024-05-27T06:45:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.074+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe75c064e10 con 0x7fe75c108b80 2024-05-27T06:45:30.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.967+0000 7fe759600700 1 -- 172.21.15.16:0/3455136836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe74c0282c0 con 0x7fe75c108b80 2024-05-27T06:45:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.971+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 msgr2=0x7fe73c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.971+0000 7fe763000700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 0x7fe73c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.971+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 msgr2=0x7fe75c19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.971+0000 7fe763000700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b5c0 tx=0x7fe74c005710 comp rx=0 tx=0).stop 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 shutdown_connections 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe73c03e510 0x7fe73c040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 --2- 172.21.15.16:0/3455136836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c108b80 0x7fe75c19f580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 >> 172.21.15.16:0/3455136836 conn(0x7fe75c0fde20 msgr2=0x7fe75c105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 shutdown_connections 2024-05-27T06:45:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:45:30.972+0000 7fe763000700 1 -- 172.21.15.16:0/3455136836 wait complete. 2024-05-27T06:45:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-05-27T06:45:31.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:30 smithi016 ceph-mon[31415]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:31.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:30 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T06:45:31.753 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-05-27T06:45:31.753 INFO:tasks.cephadm:Adding mon.b on smithi069 2024-05-27T06:45:31.753 INFO:tasks.cephadm:Adding mon.c on smithi164 2024-05-27T06:45:31.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c' 2024-05-27T06:45:32.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:31 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T06:45:32.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:31 smithi016 ceph-mon[31415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:32.504 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T06:45:32.971 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T06:45:33.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:32 smithi016 ceph-mon[31415]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:34.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:34 smithi016 ceph-mon[31415]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 -- 172.21.15.164:0/716170358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 msgr2=0x7f0338075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 --2- 172.21.15.164:0/716170358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f0338075b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0328009910 tx=0x7f0328009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 -- 172.21.15.164:0/716170358 shutdown_connections 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 --2- 172.21.15.164:0/716170358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f0338075b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 -- 172.21.15.164:0/716170358 >> 172.21.15.164:0/716170358 conn(0x7f03380ff250 msgr2=0x7f0338101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 -- 172.21.15.164:0/716170358 shutdown_connections 2024-05-27T06:45:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.269+0000 7f033e200700 1 -- 172.21.15.164:0/716170358 wait complete. 2024-05-27T06:45:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.270+0000 7f033e200700 1 Processor -- start 2024-05-27T06:45:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.270+0000 7f033e200700 1 -- start start 2024-05-27T06:45:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.271+0000 7f033e200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.271+0000 7f033e200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f033810bc70 con 0x7f03380776f0 2024-05-27T06:45:35.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.271+0000 7f0337400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:35.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.271+0000 7f0337400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.164:47776/0 (socket says 172.21.15.164:47776) 2024-05-27T06:45:35.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.271+0000 7f0337400700 1 -- 172.21.15.164:0/1271120599 learned_addr learned my addr 172.21.15.164:0/1271120599 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:45:35.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.272+0000 7f0337400700 1 -- 172.21.15.164:0/1271120599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033819bd80 con 0x7f03380776f0 2024-05-27T06:45:35.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.273+0000 7f0337400700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0328005790 tx=0x7f0328004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.273+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0328027070 con 0x7f03380776f0 2024-05-27T06:45:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.273+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033819c010 con 0x7f03380776f0 2024-05-27T06:45:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.273+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033819ec20 con 0x7f03380776f0 2024-05-27T06:45:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.273+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0328004830 con 0x7f03380776f0 2024-05-27T06:45:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.274+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0328031060 con 0x7f03380776f0 2024-05-27T06:45:35.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.274+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318005360 con 0x7f03380776f0 2024-05-27T06:45:35.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.275+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f032802a070 con 0x7f03380776f0 2024-05-27T06:45:35.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.276+0000 7f032fe00700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 0x7f031c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:35.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.276+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0328053830 con 0x7f03380776f0 2024-05-27T06:45:35.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.277+0000 7f0336a00700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 0x7f031c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:35.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.278+0000 7f0336a00700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 0x7f031c040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:35.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.294+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0328013070 con 0x7f03380776f0 2024-05-27T06:45:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:35 smithi016 ceph-mon[31415]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:35 smithi016 ceph-mon[31415]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:35 smithi016 ceph-mon[31415]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:35.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.547+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}) -- 0x7f0318000cb0 con 0x7f031c03e570 2024-05-27T06:45:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.564+0000 7f032fe00700 1 -- 172.21.15.164:0/1271120599 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0318000cb0 con 0x7f031c03e570 2024-05-27T06:45:35.557 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2024-05-27T06:45:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.567+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 msgr2=0x7f031c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.567+0000 7f033e200700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 0x7f031c040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.567+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 msgr2=0x7f033819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.567+0000 7f033e200700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0328005790 tx=0x7f0328004f60 comp rx=0 tx=0).stop 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 shutdown_connections 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f031c03e570 0x7f031c040a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 --2- 172.21.15.164:0/1271120599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380776f0 0x7f033819b840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 >> 172.21.15.164:0/1271120599 conn(0x7f03380ff250 msgr2=0x7f03380fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 shutdown_connections 2024-05-27T06:45:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:35.568+0000 7f033e200700 1 -- 172.21.15.164:0/1271120599 wait complete. 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:36.489 DEBUG:teuthology.orchestra.run.smithi069:mon.b> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.b.service 2024-05-27T06:45:36.497 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.c.service 2024-05-27T06:45:36.502 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T06:45:36.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T06:45:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:37 smithi016 ceph-mon[31415]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:37 smithi016 ceph-mon[31415]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c;count:3 2024-05-27T06:45:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:37 smithi016 ceph-mon[31415]: Deploying daemon mon.c on smithi164 2024-05-27T06:45:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:37 smithi016 ceph-mon[31415]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:37.853 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:39 smithi016 ceph-mon[31415]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 -- 172.21.15.164:0/3218352375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 msgr2=0x7fe9d4105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/3218352375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d4105150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009910 tx=0x7fe9c4009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 -- 172.21.15.164:0/3218352375 shutdown_connections 2024-05-27T06:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/3218352375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d4105150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 -- 172.21.15.164:0/3218352375 >> 172.21.15.164:0/3218352375 conn(0x7fe9d40785a0 msgr2=0x7fe9d4078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 -- 172.21.15.164:0/3218352375 shutdown_connections 2024-05-27T06:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.185+0000 7fe9d8e00700 1 -- 172.21.15.164:0/3218352375 wait complete. 2024-05-27T06:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.187+0000 7fe9d8e00700 1 Processor -- start 2024-05-27T06:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.187+0000 7fe9d8e00700 1 -- start start 2024-05-27T06:45:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.187+0000 7fe9d8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.187+0000 7fe9d8e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d410dfe0 con 0x7fe9d4104d40 2024-05-27T06:45:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.188+0000 7fe9d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.188+0000 7fe9d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.164:52940/0 (socket says 172.21.15.164:52940) 2024-05-27T06:45:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.188+0000 7fe9d1e00700 1 -- 172.21.15.164:0/4160463657 learned_addr learned my addr 172.21.15.164:0/4160463657 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.188+0000 7fe9d1e00700 1 -- 172.21.15.164:0/4160463657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d41a1c50 con 0x7fe9d4104d40 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.189+0000 7fe9d1e00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006010 tx=0x7fe9c4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.189+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c4026070 con 0x7fe9d4104d40 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.189+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d41a1ee0 con 0x7fe9d4104d40 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.189+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d419b930 con 0x7fe9d4104d40 2024-05-27T06:45:41.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.190+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9c400b750 con 0x7fe9d4104d40 2024-05-27T06:45:41.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.190+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c40038a0 con 0x7fe9d4104d40 2024-05-27T06:45:41.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.190+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d406b6c0 con 0x7fe9d4104d40 2024-05-27T06:45:41.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.191+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe9c4029070 con 0x7fe9d4104d40 2024-05-27T06:45:41.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.192+0000 7fe9caa00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 0x7fe9b8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:41.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.192+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe9c401e070 con 0x7fe9d4104d40 2024-05-27T06:45:41.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.193+0000 7fe9d1400700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 0x7fe9b8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:41.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.194+0000 7fe9d1400700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 0x7fe9b8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:41.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.210+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9c4013070 con 0x7fe9d4104d40 2024-05-27T06:45:41.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:41 smithi016 ceph-mon[31415]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:41.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.609+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe9d419ce60 con 0x7fe9d4104d40 2024-05-27T06:45:41.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.610+0000 7fe9caa00700 1 -- 172.21.15.164:0/4160463657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe9c40163a0 con 0x7fe9d4104d40 2024-05-27T06:45:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:45:41.603 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","modified":"2024-05-27T06:43:10.303844Z","created":"2024-05-27T06:43:10.303844Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 msgr2=0x7fe9b8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 0x7fe9b8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0006e40 comp rx=0 tx=0).stop 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 msgr2=0x7fe9d41a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006010 tx=0x7fe9c4005710 comp rx=0 tx=0).stop 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 shutdown_connections 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9b803e510 0x7fe9b8040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 --2- 172.21.15.164:0/4160463657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d4104d40 0x7fe9d41a1710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 >> 172.21.15.164:0/4160463657 conn(0x7fe9d40785a0 msgr2=0x7fe9d4109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 shutdown_connections 2024-05-27T06:45:41.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:41.613+0000 7fe9d8e00700 1 -- 172.21.15.164:0/4160463657 wait complete. 2024-05-27T06:45:41.607 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2024-05-27T06:45:42.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:42 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/4160463657' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:42.810 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 podman[35308]: 2024-05-27 06:45:42.554951571 +0000 UTC m=+0.166586651 container create 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T06:45:42.810 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 podman[35308]: 2024-05-27 06:45:42.816242248 +0000 UTC m=+0.427877382 container init 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD) 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 podman[35308]: 2024-05-27 06:45:42.825539271 +0000 UTC m=+0.437174317 container start 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 bash[35308]: 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 systemd[1]: Started Ceph mon.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T06:45:43.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: pidfile_write: ignore empty --pid-file 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: load: jerasure load: lrc 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: RocksDB version: 7.9.2 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Git sha 0 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: DB SUMMARY 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: DB Session ID: DWXU2BPGJONSMVYL05LG 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: CURRENT file: CURRENT 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.error_if_exists: 0 2024-05-27T06:45:43.088 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.create_if_missing: 0 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.paranoid_checks: 1 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.env: 0x555bcc931500 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.info_log: 0x555bdae8c040 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.statistics: (nil) 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.use_fsync: 0 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_log_file_size: 0 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T06:45:43.089 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_fallocate: 1 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.use_direct_reads: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T06:45:43.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.db_log_dir: 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.wal_dir: 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.write_buffer_manager: 0x555bdad568c0 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T06:45:43.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.unordered_write: 0 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.row_cache: None 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.wal_filter: None 2024-05-27T06:45:43.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.two_write_queues: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.wal_compression: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.atomic_flush: 0 2024-05-27T06:45:43.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.log_readahead_size: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_background_jobs: 2 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_background_compactions: -1 2024-05-27T06:45:43.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_subcompactions: 1 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_open_files: -1 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T06:45:43.095 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_background_flushes: -1 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Compression algorithms supported: 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kZSTD supported: 0 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kXpressCompression supported: 0 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kZlibCompression supported: 1 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kSnappyCompression supported: 1 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kLZ4Compression supported: 1 2024-05-27T06:45:43.096 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: kBZip2Compression supported: 0 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.merge_operator: 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_filter: None 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555bdae8c200) 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-05-27T06:45:43.097 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_type: 0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_index_type: 0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_shortening: 1 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: checksum: 4 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: no_block_cache: 0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache: 0x555bdad891f0 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_options: 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: capacity : 536870912 2024-05-27T06:45:43.098 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_shard_bits : 4 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: strict_capacity_limit : 0 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_compressed: (nil) 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: persistent_cache: (nil) 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size: 4096 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size_deviation: 10 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_restart_interval: 16 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_block_restart_interval: 1 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: metadata_block_size: 4096 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: partition_filters: 0 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: use_delta_encoding: 1 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: filter_policy: bloomfilter 2024-05-27T06:45:43.099 INFO:journalctl@ceph.mon.c.smithi164.stdout: whole_key_filtering: 1 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: verify_compression: 0 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: format_version: 5 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_index_compression: 1 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_align: 0 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: max_auto_readahead_size: 262144 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: prepopulate_block_cache: 0 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: initial_auto_readahead_size: 8192 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T06:45:43.100 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T06:45:43.101 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression: NoCompression 2024-05-27T06:45:43.101 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T06:45:43.101 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T06:45:43.101 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.num_levels: 7 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T06:45:43.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T06:45:43.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T06:45:43.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T06:45:43.105 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.inplace_update_support: 0 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T06:45:43.106 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.bloom_locality: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.max_successive_merges: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.ttl: 2592000 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enable_blob_files: false 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.min_blob_size: 0 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T06:45:43.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 65cf444d-31db-48f5-b469-3e4893a9aa7e 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792342965548, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792342966904, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716792342, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "65cf444d-31db-48f5-b469-3e4893a9aa7e", "db_session_id": "DWXU2BPGJONSMVYL05LG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792342967226, "job": 1, "event": "recovery_finished"} 2024-05-27T06:45:43.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555bdad6dc00 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: DB pointer 0x555bdfb60000 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** DB Stats ** 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:43.109 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:43.110 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache BinnedLRUCache@0x555bdad891f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T06:45:43.111 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: starting mon.c rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:42 smithi164 ceph-mon[35327]: mon.c@-1(???) e0 preinit fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).mds e1 new map 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: e1 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: legacy client fscid: -1 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout: No filesystems configured 2024-05-27T06:45:43.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Deploying cephadm binary to smithi069 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Added host smithi069 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:43.115 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Deploying cephadm binary to smithi164 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Added host smithi164 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.116 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:43.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c;count:3 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: Deploying daemon mon.c on smithi164 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/4160463657' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:43.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:43 smithi164 ceph-mon[35327]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T06:45:43.313 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T06:45:43.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T06:45:43.748 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: mon.a calling monitor election 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: mon.c calling monitor election 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: monmap epoch 2 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: last_changed 2024-05-27T06:45:43.031030+0000 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: min_mon_release 19 (squid) 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: election_strategy: 1 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: fsmap 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: mgrmap e13: x(active, since 70s) 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: overall HEALTH_OK 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: mon.a calling monitor election 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: mon.c calling monitor election 2024-05-27T06:45:48.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: monmap epoch 2 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: last_changed 2024-05-27T06:45:43.031030+0000 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: min_mon_release 19 (squid) 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: election_strategy: 1 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: fsmap 2024-05-27T06:45:48.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:48.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: mgrmap e13: x(active, since 70s) 2024-05-27T06:45:48.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: overall HEALTH_OK 2024-05-27T06:45:48.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:48.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:48.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:48.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.762+0000 7f28c2e00700 1 -- 172.21.15.164:0/317619660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 msgr2=0x7f28a4005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.762+0000 7f28c2e00700 1 --2- 172.21.15.164:0/317619660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28a4005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f28bc075a80 tx=0x7f28b000feb0 comp rx=0 tx=0).stop 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.763+0000 7f28c2e00700 1 -- 172.21.15.164:0/317619660 shutdown_connections 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.763+0000 7f28c2e00700 1 --2- 172.21.15.164:0/317619660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28a4005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.763+0000 7f28c2e00700 1 -- 172.21.15.164:0/317619660 >> 172.21.15.164:0/317619660 conn(0x7f28bc0ff570 msgr2=0x7f28bc1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.763+0000 7f28c2e00700 1 -- 172.21.15.164:0/317619660 shutdown_connections 2024-05-27T06:45:48.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.763+0000 7f28c2e00700 1 -- 172.21.15.164:0/317619660 wait complete. 2024-05-27T06:45:48.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.765+0000 7f28c2e00700 1 Processor -- start 2024-05-27T06:45:48.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.766+0000 7f28c2e00700 1 -- start start 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.767+0000 7f28c2e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.767+0000 7f28c2e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28bc1a1ba0 0x7f28bc19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.767+0000 7f28c2e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10da90 con 0x7f28a40035c0 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.767+0000 7f28c2e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10d910 con 0x7f28bc1a1ba0 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.768+0000 7f28c1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.768+0000 7f28c1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.164:53030/0 (socket says 172.21.15.164:53030) 2024-05-27T06:45:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.768+0000 7f28c1a00700 1 -- 172.21.15.164:0/745716158 learned_addr learned my addr 172.21.15.164:0/745716158 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:45:48.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.769+0000 7f28c1000700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28bc1a1ba0 0x7f28bc19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:48.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.769+0000 7f28c1a00700 1 -- 172.21.15.164:0/745716158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28bc1a1ba0 msgr2=0x7f28bc19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:48.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.769+0000 7f28c1a00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28bc1a1ba0 0x7f28bc19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:48.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.769+0000 7f28c1a00700 1 -- 172.21.15.164:0/745716158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc1a2170 con 0x7f28a40035c0 2024-05-27T06:45:48.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28c1a00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28b0010010 tx=0x7f28b0005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f28b0013b00 con 0x7f28a40035c0 2024-05-27T06:45:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc19be20 con 0x7f28a40035c0 2024-05-27T06:45:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28b00044c0 con 0x7f28a40035c0 2024-05-27T06:45:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f28b0003c70 con 0x7f28a40035c0 2024-05-27T06:45:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.770+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc19c3b0 con 0x7f28a40035c0 2024-05-27T06:45:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.772+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28bc077290 con 0x7f28a40035c0 2024-05-27T06:45:48.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.772+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f28b0023070 con 0x7f28a40035c0 2024-05-27T06:45:48.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.773+0000 7f28ba000700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 0x7f289c0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:48.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.773+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f28b0020050 con 0x7f28a40035c0 2024-05-27T06:45:48.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.774+0000 7f28c1000700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 0x7f289c0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:48.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.775+0000 7f28c1000700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 0x7f289c0407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28bc10bc20 tx=0x7f28a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:48.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:48.791+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28b0028020 con 0x7f28a40035c0 2024-05-27T06:45:49.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.124+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f28bc19cde0 con 0x7f28a40035c0 2024-05-27T06:45:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.125+0000 7f28ba000700 1 -- 172.21.15.164:0/745716158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f28b002c3f0 con 0x7f28a40035c0 2024-05-27T06:45:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:45:49.119 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","modified":"2024-05-27T06:45:43.031030Z","created":"2024-05-27T06:43:10.303844Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T06:45:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 msgr2=0x7f289c0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 0x7f289c0407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28bc10bc20 tx=0x7f28a8008040 comp rx=0 tx=0).stop 2024-05-27T06:45:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 msgr2=0x7f28bc1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28b0010010 tx=0x7f28b0005350 comp rx=0 tx=0).stop 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 shutdown_connections 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f289c03e2b0 0x7f289c0407c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40035c0 0x7f28bc1a1660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 --2- 172.21.15.164:0/745716158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28bc1a1ba0 0x7f28bc19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.128+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 >> 172.21.15.164:0/745716158 conn(0x7f28bc0ff570 msgr2=0x7f28bc102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.129+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 shutdown_connections 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:49.129+0000 7f28c2e00700 1 -- 172.21.15.164:0/745716158 wait complete. 2024-05-27T06:45:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2024-05-27T06:45:49.454 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:49 smithi164 ceph-mon[35327]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:49.454 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:49 smithi164 ceph-mon[35327]: Deploying daemon mon.b on smithi069 2024-05-27T06:45:49.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:49.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:49 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/745716158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:49 smithi016 ceph-mon[31415]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:49 smithi016 ceph-mon[31415]: Deploying daemon mon.b on smithi069 2024-05-27T06:45:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:49 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/745716158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:51.041 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T06:45:51.042 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T06:45:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:45:51.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:51 smithi164 ceph-mon[35327]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:51 smithi016 ceph-mon[31415]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:53 smithi164 ceph-mon[35327]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 -- 172.21.15.164:0/511882636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1354104e20 msgr2=0x7f1354105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:53.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 --2- 172.21.15.164:0/511882636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1354104e20 0x7f1354105230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1348009910 tx=0x7f1348009c20 comp rx=0 tx=0).stop 2024-05-27T06:45:53.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 -- 172.21.15.164:0/511882636 shutdown_connections 2024-05-27T06:45:53.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 --2- 172.21.15.164:0/511882636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1354104e20 0x7f1354105230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:53.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 -- 172.21.15.164:0/511882636 >> 172.21.15.164:0/511882636 conn(0x7f13540fe650 msgr2=0x7f1354100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:53.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 -- 172.21.15.164:0/511882636 shutdown_connections 2024-05-27T06:45:53.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.478+0000 7f135b800700 1 -- 172.21.15.164:0/511882636 wait complete. 2024-05-27T06:45:53.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.480+0000 7f135b800700 1 Processor -- start 2024-05-27T06:45:53.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.480+0000 7f135b800700 1 -- start start 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f135b800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f135b800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135419fca0 0x7f1354199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f135b800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f135410e0c0 con 0x7f135419fca0 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f135b800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f135410df40 con 0x7f1354104e20 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f1358e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f1358e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57346/0 (socket says 172.21.15.164:57346) 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f1358e00700 1 -- 172.21.15.164:0/1071282683 learned_addr learned my addr 172.21.15.164:0/1071282683 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:45:53.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.481+0000 7f1353e00700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135419fca0 0x7f1354199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:53.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.482+0000 7f1358e00700 1 -- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135419fca0 msgr2=0x7f1354199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:53.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.482+0000 7f1358e00700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135419fca0 0x7f1354199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:53.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.482+0000 7f1358e00700 1 -- 172.21.15.164:0/1071282683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541a0270 con 0x7f1354104e20 2024-05-27T06:45:53.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.482+0000 7f1358e00700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1348005b40 tx=0x7f13480056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.482+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f134800b960 con 0x7f1354104e20 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.483+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1348022470 con 0x7f1354104e20 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.483+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f134800f660 con 0x7f1354104e20 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.483+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1354199ec0 con 0x7f1354104e20 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.483+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135419a450 con 0x7f1354104e20 2024-05-27T06:45:53.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.484+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135419a7c0 con 0x7f1354104e20 2024-05-27T06:45:53.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.484+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f134802a070 con 0x7f1354104e20 2024-05-27T06:45:53.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.485+0000 7f1351600700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 0x7f133c040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:45:53.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.485+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f134801e030 con 0x7f1354104e20 2024-05-27T06:45:53.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.486+0000 7f1353e00700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 0x7f133c040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:45:53.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.487+0000 7f1353e00700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 0x7f133c040810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1354107040 tx=0x7f1340009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:45:53.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.492+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1348055e20 con 0x7f1354104e20 2024-05-27T06:45:53.597 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 podman[35256]: 2024-05-27 06:45:53.603985285 +0000 UTC m=+0.444880585 container init d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T06:45:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:53 smithi016 ceph-mon[31415]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.808+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134802baa0 con 0x7f1354104e20 2024-05-27T06:45:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:53.816+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f135419af50 con 0x7f1354104e20 2024-05-27T06:45:53.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 podman[35256]: 2024-05-27 06:45:53.61398846 +0000 UTC m=+0.454883717 container start d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T06:45:53.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 bash[35256]: d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b 2024-05-27T06:45:53.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 systemd[1]: Started Ceph mon.b for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:45:53.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T06:45:53.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pidfile_write: ignore empty --pid-file 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: load: jerasure load: lrc 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: RocksDB version: 7.9.2 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Git sha 0 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: DB SUMMARY 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: DB Session ID: R9U8I6AR5L77PRDZF445 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: CURRENT file: CURRENT 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-27T06:45:53.872 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.error_if_exists: 0 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.create_if_missing: 0 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.paranoid_checks: 1 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.env: 0x559f8d48f500 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.info_log: 0x559fb6b18040 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.statistics: (nil) 2024-05-27T06:45:53.873 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.use_fsync: 0 2024-05-27T06:45:53.874 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_log_file_size: 0 2024-05-27T06:45:53.874 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T06:45:53.874 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T06:45:53.874 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_fallocate: 1 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.use_direct_reads: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.db_log_dir: 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.wal_dir: 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T06:45:53.876 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.write_buffer_manager: 0x559fb69e28c0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T06:45:53.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.unordered_write: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.row_cache: None 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.wal_filter: None 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.two_write_queues: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.wal_compression: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.atomic_flush: 0 2024-05-27T06:45:53.878 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.log_readahead_size: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_background_jobs: 2 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_background_compactions: -1 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_subcompactions: 1 2024-05-27T06:45:53.879 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_open_files: -1 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_background_flushes: -1 2024-05-27T06:45:53.880 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Compression algorithms supported: 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kZSTD supported: 0 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kXpressCompression supported: 0 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kZlibCompression supported: 1 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kSnappyCompression supported: 1 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kLZ4Compression supported: 1 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: kBZip2Compression supported: 0 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.merge_operator: 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_filter: None 2024-05-27T06:45:53.881 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559fb6b18200) 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_type: 0 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: data_block_index_type: 0 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_shortening: 1 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T06:45:53.882 INFO:journalctl@ceph.mon.b.smithi069.stdout: checksum: 4 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: no_block_cache: 0 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache: 0x559fb6a151f0 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_options: 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: capacity : 536870912 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: num_shard_bits : 4 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: strict_capacity_limit : 0 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_compressed: (nil) 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: persistent_cache: (nil) 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_size: 4096 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_size_deviation: 10 2024-05-27T06:45:53.883 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_restart_interval: 16 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_block_restart_interval: 1 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: metadata_block_size: 4096 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: partition_filters: 0 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: use_delta_encoding: 1 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: filter_policy: bloomfilter 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: whole_key_filtering: 1 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: verify_compression: 0 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: format_version: 5 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: enable_index_compression: 1 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_align: 0 2024-05-27T06:45:53.884 INFO:journalctl@ceph.mon.b.smithi069.stdout: max_auto_readahead_size: 262144 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout: prepopulate_block_cache: 0 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout: initial_auto_readahead_size: 8192 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression: NoCompression 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.num_levels: 7 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T06:45:53.885 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T06:45:53.886 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T06:45:53.887 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T06:45:53.888 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.inplace_update_support: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.bloom_locality: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.max_successive_merges: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T06:45:53.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.ttl: 2592000 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enable_blob_files: false 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.min_blob_size: 0 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T06:45:53.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ec1326a1-7102-4142-b939-baca7fae2b50 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792353745251, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792353746508, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716792353, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ec1326a1-7102-4142-b939-baca7fae2b50", "db_session_id": "R9U8I6AR5L77PRDZF445", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716792353746851, "job": 1, "event": "recovery_finished"} 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559fb69f9c00 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: DB pointer 0x559fbb7ec000 2024-05-27T06:45:53.891 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** DB Stats ** 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** Compaction Stats [default] ** 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:53.892 INFO:journalctl@ceph.mon.b.smithi069.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** Compaction Stats [default] ** 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T06:45:53.893 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: Block cache BinnedLRUCache@0x559fb6a151f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: using public_addr v2:172.21.15.69:0/0 -> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: starting mon.b rank -1 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:53.894 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(???) e0 preinit fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).mds e1 new map 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: e1 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: legacy client fscid: -1 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout: No filesystems configured 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T06:45:53.895 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Deploying cephadm binary to smithi069 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Added host smithi069 2024-05-27T06:45:53.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:53.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Deploying cephadm binary to smithi164 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Added host smithi164 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3455136836' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c;count:3 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Deploying daemon mon.c on smithi164 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/4160463657' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.a calling monitor election 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.c calling monitor election 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: monmap epoch 2 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: last_changed 2024-05-27T06:45:43.031030+0000 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: min_mon_release 19 (squid) 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: election_strategy: 1 2024-05-27T06:45:53.902 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: fsmap 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mgrmap e13: x(active, since 70s) 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: overall HEALTH_OK 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: Deploying daemon mon.b on smithi069 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/745716158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:53.903 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:53 smithi069 ceph-mon[35276]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-27T06:45:58.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.832+0000 7f1351600700 1 -- 172.21.15.164:0/1071282683 <== mon.1 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f1348013070 con 0x7f1354104e20 2024-05-27T06:45:58.826 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-27T06:45:58.826 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","modified":"2024-05-27T06:45:53.802559Z","created":"2024-05-27T06:43:10.303844Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 msgr2=0x7f133c040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 0x7f133c040810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1354107040 tx=0x7f1340009210 comp rx=0 tx=0).stop 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 msgr2=0x7f135419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1348005b40 tx=0x7f13480056d0 comp rx=0 tx=0).stop 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 shutdown_connections 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f133c03e300 0x7f133c040810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1354104e20 0x7f135419f760 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 --2- 172.21.15.164:0/1071282683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135419fca0 0x7f1354199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:45:58.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 >> 172.21.15.164:0/1071282683 conn(0x7f13540fe650 msgr2=0x7f135410a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:45:58.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 shutdown_connections 2024-05-27T06:45:58.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:45:58.836+0000 7f135b800700 1 -- 172.21.15.164:0/1071282683 wait complete. 2024-05-27T06:45:58.831 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2024-05-27T06:45:59.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:59.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: mon.a calling monitor election 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: mon.c calling monitor election 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/1071282683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: mon.b calling monitor election 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: monmap epoch 3 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: last_changed 2024-05-27T06:45:53.802559+0000 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: min_mon_release 19 (squid) 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: election_strategy: 1 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: fsmap 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: mgrmap e13: x(active, since 80s) 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: overall HEALTH_OK 2024-05-27T06:45:59.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.086 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.086 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.336 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: mon.a calling monitor election 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: mon.c calling monitor election 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/1071282683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: mon.b calling monitor election 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: monmap epoch 3 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: last_changed 2024-05-27T06:45:53.802559+0000 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: min_mon_release 19 (squid) 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: election_strategy: 1 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:59.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: fsmap 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: mgrmap e13: x(active, since 80s) 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: overall HEALTH_OK 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.339 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:45:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: mon.a calling monitor election 2024-05-27T06:45:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: mon.c calling monitor election 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/1071282683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: mon.b calling monitor election 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:45:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: monmap epoch 3 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: last_changed 2024-05-27T06:45:53.802559+0000 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: created 2024-05-27T06:43:10.303844+0000 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: min_mon_release 19 (squid) 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: election_strategy: 1 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-27T06:45:59.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: fsmap 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: mgrmap e13: x(active, since 80s) 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: overall HEALTH_OK 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:45:59.562 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T06:45:59.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T06:46:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:46:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:45:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:46:00.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:45:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:46:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:45:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:46:01.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:00 smithi016 ceph-mon[31415]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:01.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:00 smithi164 ceph-mon[35327]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:00 smithi069 ceph-mon[35276]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 -- 172.21.15.16:0/857566644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c103de0 msgr2=0x7fa72c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 --2- 172.21.15.16:0/857566644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c103de0 0x7fa72c1041f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c009c20 comp rx=0 tx=0).stop 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 -- 172.21.15.16:0/857566644 shutdown_connections 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 --2- 172.21.15.16:0/857566644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c103de0 0x7fa72c1041f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 -- 172.21.15.16:0/857566644 >> 172.21.15.16:0/857566644 conn(0x7fa72c0ff690 msgr2=0x7fa72c101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 -- 172.21.15.16:0/857566644 shutdown_connections 2024-05-27T06:46:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.219+0000 7fa732400700 1 -- 172.21.15.16:0/857566644 wait complete. 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.220+0000 7fa732400700 1 Processor -- start 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.220+0000 7fa732400700 1 -- start start 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa72c103de0 0x7fa72c1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c1a1f00 0x7fa72c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10df20 con 0x7fa72c19c090 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10dda0 con 0x7fa72c1a1f00 2024-05-27T06:46:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa732400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10e0a0 con 0x7fa72c103de0 2024-05-27T06:46:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa72be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa72be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58884/0 (socket says 172.21.15.16:58884) 2024-05-27T06:46:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa72be00700 1 -- 172.21.15.16:0/3361470657 learned_addr learned my addr 172.21.15.16:0/3361470657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:46:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.221+0000 7fa72b400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa72c103de0 0x7fa72c1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72aa00700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c1a1f00 0x7fa72c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 -- 172.21.15.16:0/3361470657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa72c103de0 msgr2=0x7fa72c1a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa72c103de0 0x7fa72c1a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 -- 172.21.15.16:0/3361470657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c1a1f00 msgr2=0x7fa72c19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c1a1f00 0x7fa72c19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 -- 172.21.15.16:0/3361470657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c1a8770 con 0x7fa72c19c090 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa72be00700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa72400ba50 tx=0x7fa72400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724019070 con 0x7fa72c19c090 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa72400ce40 con 0x7fa72c19c090 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.222+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c1a8a60 con 0x7fa72c19c090 2024-05-27T06:46:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.223+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1a8f70 con 0x7fa72c19c090 2024-05-27T06:46:02.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.223+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724007350 con 0x7fa72c19c090 2024-05-27T06:46:02.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.224+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c005330 con 0x7fa72c19c090 2024-05-27T06:46:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.224+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa72401c070 con 0x7fa72c19c090 2024-05-27T06:46:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.225+0000 7fa723e00700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 0x7fa710040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.225+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa724051e30 con 0x7fa72c19c090 2024-05-27T06:46:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.225+0000 7fa72b400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 0x7fa710040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.226+0000 7fa72b400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 0x7fa710040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009750 tx=0x7fa71c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:02.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.231+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7240516a0 con 0x7fa72c19c090 2024-05-27T06:46:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.476+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa70c005620 con 0x7fa72c19c090 2024-05-27T06:46:02.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:46:02.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:02 smithi016 ceph-mon[31415]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:46:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.481+0000 7fa723e00700 1 -- 172.21.15.16:0/3361470657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa724013a90 con 0x7fa72c19c090 2024-05-27T06:46:02.475 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:46:02.475 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-05-27T06:46:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T06:46:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-05-27T06:46:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 msgr2=0x7fa710040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 0x7fa710040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009750 tx=0x7fa71c005fb0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 msgr2=0x7fa72c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa72400ba50 tx=0x7fa72400bd60 comp rx=0 tx=0).stop 2024-05-27T06:46:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 shutdown_connections 2024-05-27T06:46:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa71003e9a0 0x7fa710040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c19c090 0x7fa72c19c4a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa72c103de0 0x7fa72c1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 --2- 172.21.15.16:0/3361470657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c1a1f00 0x7fa72c19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 >> 172.21.15.16:0/3361470657 conn(0x7fa72c0ff690 msgr2=0x7fa72c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.485+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 shutdown_connections 2024-05-27T06:46:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:02.486+0000 7fa732400700 1 -- 172.21.15.16:0/3361470657 wait complete. 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:46:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:02 smithi164 ceph-mon[35327]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:46:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T06:46:02.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:02 smithi069 ceph-mon[35276]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T06:46:03.306 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T06:46:03.306 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:46:03.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T06:46:03.406 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:46:03.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:46:03.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:03.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3361470657' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.524 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:46:03.524 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T06:46:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3361470657' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3361470657' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:03.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:03.617 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:46:03.617 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:46:03.744 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:46:03.744 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T06:46:03.834 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:46:03.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T06:46:03.962 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-05-27T06:46:03.962 INFO:tasks.cephadm:Adding mgr.y on smithi069 2024-05-27T06:46:03.962 INFO:tasks.cephadm:Adding mgr.z on smithi164 2024-05-27T06:46:03.962 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi069=y;smithi164=z' 2024-05-27T06:46:04.420 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: Reconfiguring daemon mon.a on smithi016 2024-05-27T06:46:04.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:04 smithi016 ceph-mon[31415]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:04.629 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: Reconfiguring daemon mon.a on smithi016 2024-05-27T06:46:04.630 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:04 smithi164 ceph-mon[35327]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T06:46:04.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:04.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:04.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:04.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: Reconfiguring daemon mon.a on smithi016 2024-05-27T06:46:04.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:04 smithi069 ceph-mon[35276]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 -- 172.21.15.164:0/2105835202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 msgr2=0x7fc9f0106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 --2- 172.21.15.164:0/2105835202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f0106160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009910 tx=0x7fc9e0009c20 comp rx=0 tx=0).stop 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 -- 172.21.15.164:0/2105835202 shutdown_connections 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 --2- 172.21.15.164:0/2105835202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f0106160 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 -- 172.21.15.164:0/2105835202 >> 172.21.15.164:0/2105835202 conn(0x7fc9f01013d0 msgr2=0x7fc9f0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 -- 172.21.15.164:0/2105835202 shutdown_connections 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.665+0000 7fc9ef200700 1 -- 172.21.15.164:0/2105835202 wait complete. 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.666+0000 7fc9ef200700 1 Processor -- start 2024-05-27T06:46:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.666+0000 7fc9ef200700 1 -- start start 2024-05-27T06:46:06.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.666+0000 7fc9ef200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.666+0000 7fc9ef200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9f019be00 0x7fc9f01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.666+0000 7fc9ef200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ef200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010bbc0 con 0x7fc9f0105d50 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ef200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010ba40 con 0x7fc9f019be00 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ef200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010bd40 con 0x7fc9f01a12d0 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ee800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ee800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37816/0 (socket says 172.21.15.164:37816) 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ee800700 1 -- 172.21.15.164:0/16354433 learned_addr learned my addr 172.21.15.164:0/16354433 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ede00700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:06.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ed400700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9f019be00 0x7fc9f01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ee800700 1 -- 172.21.15.164:0/16354433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9f019be00 msgr2=0x7fc9f01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.667+0000 7fc9ee800700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9f019be00 0x7fc9f01a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ee800700 1 -- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 msgr2=0x7fc9f019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ee800700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ee800700 1 -- 172.21.15.164:0/16354433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f01a3c60 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ee800700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800ba50 tx=0x7fc9e800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8019070 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9e800ce40 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01a3f50 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8007350 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.668+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a43c0 con 0x7fc9f01a12d0 2024-05-27T06:46:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.670+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f01a46e0 con 0x7fc9f01a12d0 2024-05-27T06:46:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.670+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc9e801c070 con 0x7fc9f01a12d0 2024-05-27T06:46:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.671+0000 7fc9e6a00700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 0x7fc9d4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.671+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9e8051ec0 con 0x7fc9f01a12d0 2024-05-27T06:46:06.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.671+0000 7fc9ede00700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 0x7fc9d4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:06.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.673+0000 7fc9ede00700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 0x7fc9d4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0006010 tx=0x7fc9e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:06.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.677+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9e801de80 con 0x7fc9f01a12d0 2024-05-27T06:46:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.925+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi069=y;smithi164=z", "target": ["mon-mgr", ""]}) -- 0x7fc9f0103220 con 0x7fc9d403e9a0 2024-05-27T06:46:06.919 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:06.919 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:06.919 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: Reconfiguring mon.b (monmap changed)... 2024-05-27T06:46:06.919 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:06.919 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:06.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:06.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: Reconfiguring daemon mon.b on smithi069 2024-05-27T06:46:06.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:06 smithi164 ceph-mon[35327]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:06.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.941+0000 7fc9e6a00700 1 -- 172.21.15.164:0/16354433 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc9f0103220 con 0x7fc9d403e9a0 2024-05-27T06:46:06.934 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2024-05-27T06:46:06.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.943+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 msgr2=0x7fc9d4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.943+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 0x7fc9d4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0006010 tx=0x7fc9e001a040 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 msgr2=0x7fc9f01a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800ba50 tx=0x7fc9e800bd60 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 shutdown_connections 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9d403e9a0 0x7fc9d4040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f01a12d0 0x7fc9f01a36f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0105d50 0x7fc9f019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 --2- 172.21.15.164:0/16354433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9f019be00 0x7fc9f01a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 >> 172.21.15.164:0/16354433 conn(0x7fc9f01013d0 msgr2=0x7fc9f0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 shutdown_connections 2024-05-27T06:46:06.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:46:06.944+0000 7fc9ef200700 1 -- 172.21.15.164:0/16354433 wait complete. 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: Reconfiguring mon.b (monmap changed)... 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: Reconfiguring daemon mon.b on smithi069 2024-05-27T06:46:06.954 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:06 smithi069 ceph-mon[35276]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: Reconfiguring mon.b (monmap changed)... 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: Reconfiguring daemon mon.b on smithi069 2024-05-27T06:46:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:06 smithi016 ceph-mon[31415]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:07.712 DEBUG:teuthology.orchestra.run.smithi069:mgr.y> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.y.service 2024-05-27T06:46:07.722 DEBUG:teuthology.orchestra.run.smithi164:mgr.z> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.z.service 2024-05-27T06:46:07.729 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T06:46:07.729 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:46:07.729 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T06:46:07.790 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T06:46:07.790 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-27 06:44:52.298460318 +0000 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-27 06:38:10.804467683 +0000 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-27 06:38:10.804467683 +0000 2024-05-27T06:46:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-27 06:38:10.804467683 +0000 2024-05-27T06:46:07.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T06:46:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-27T06:46:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-27T06:46:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00030898 s, 1.7 MB/s 2024-05-27T06:46:07.937 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T06:46:07.992 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-05-27T06:46:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-27 06:44:52.374459148 +0000 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-27 06:38:11.426458096 +0000 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-27 06:38:11.426458096 +0000 2024-05-27T06:46:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-27 06:38:11.426458096 +0000 2024-05-27T06:46:08.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T06:46:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-27T06:46:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-27T06:46:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000299248 s, 1.7 MB/s 2024-05-27T06:46:08.169 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T06:46:08.226 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-05-27T06:46:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-27 06:44:52.441458117 +0000 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-27 06:38:12.040448633 +0000 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-27 06:38:12.040448633 +0000 2024-05-27T06:46:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-27 06:38:12.040448633 +0000 2024-05-27T06:46:08.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi069=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: Saving service mgr spec with placement smithi016=x;smithi069=y;smithi164=z;count:3 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:08.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi069=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: Saving service mgr spec with placement smithi016=x;smithi069=y;smithi164=z;count:3 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:08.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-27T06:46:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-27T06:46:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000255868 s, 2.0 MB/s 2024-05-27T06:46:08.356 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi069=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: Saving service mgr spec with placement smithi016=x;smithi069=y;smithi164=z;count:3 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T06:46:08.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:08.410 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-05-27T06:46:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T06:46:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:08.460 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-27 06:44:52.525456824 +0000 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-27 06:38:12.628439570 +0000 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-27 06:38:12.628439570 +0000 2024-05-27T06:46:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-27 06:38:12.628439570 +0000 2024-05-27T06:46:08.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T06:46:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-27T06:46:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-27T06:46:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000307488 s, 1.7 MB/s 2024-05-27T06:46:08.545 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T06:46:08.603 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-27T06:46:08.603 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T06:46:08.662 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T06:46:08.663 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-27 06:45:52.768186814 +0000 2024-05-27T06:46:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-27 06:38:22.136114171 +0000 2024-05-27T06:46:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-27 06:38:22.136114171 +0000 2024-05-27T06:46:08.714 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-27 06:38:22.136114171 +0000 2024-05-27T06:46:08.714 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T06:46:08.786 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-27T06:46:08.786 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-27T06:46:08.786 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000390322 s, 1.3 MB/s 2024-05-27T06:46:08.789 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T06:46:08.841 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-27 06:45:52.769186792 +0000 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-27 06:38:22.711101204 +0000 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-27 06:38:22.711101204 +0000 2024-05-27T06:46:08.889 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-27 06:38:22.711101204 +0000 2024-05-27T06:46:08.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T06:46:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-27T06:46:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-27T06:46:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000301682 s, 1.7 MB/s 2024-05-27T06:46:08.971 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T06:46:09.022 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-05-27T06:46:09.036 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:08 smithi164 ceph-mon[35327]: Reconfiguring mon.c (monmap changed)... 2024-05-27T06:46:09.036 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:08 smithi164 ceph-mon[35327]: Reconfiguring daemon mon.c on smithi164 2024-05-27T06:46:09.036 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:08 smithi164 ceph-mon[35327]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-27 06:45:52.770186769 +0000 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-27 06:38:23.292088102 +0000 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-27 06:38:23.292088102 +0000 2024-05-27T06:46:09.068 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-27 06:38:23.292088102 +0000 2024-05-27T06:46:09.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T06:46:09.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:08 smithi069 ceph-mon[35276]: Reconfiguring mon.c (monmap changed)... 2024-05-27T06:46:09.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:08 smithi069 ceph-mon[35276]: Reconfiguring daemon mon.c on smithi164 2024-05-27T06:46:09.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:08 smithi069 ceph-mon[35276]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-27T06:46:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-27T06:46:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.00036424 s, 1.4 MB/s 2024-05-27T06:46:09.194 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T06:46:09.293 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-05-27T06:46:09.344 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T06:46:09.344 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:09.344 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-27T06:46:09.344 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:09.344 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:09.345 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-27 06:45:52.771186747 +0000 2024-05-27T06:46:09.345 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-27 06:38:23.898074436 +0000 2024-05-27T06:46:09.345 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-27 06:38:23.898074436 +0000 2024-05-27T06:46:09.345 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-27 06:38:23.898074436 +0000 2024-05-27T06:46:09.345 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T06:46:09.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:08 smithi016 ceph-mon[31415]: Reconfiguring mon.c (monmap changed)... 2024-05-27T06:46:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:08 smithi016 ceph-mon[31415]: Reconfiguring daemon mon.c on smithi164 2024-05-27T06:46:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:08 smithi016 ceph-mon[31415]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:09.419 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-27T06:46:09.420 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-27T06:46:09.420 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.00037373 s, 1.4 MB/s 2024-05-27T06:46:09.423 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T06:46:09.473 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-27T06:46:09.473 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T06:46:09.531 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T06:46:09.531 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-27 06:45:41.756062691 +0000 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-27 06:38:14.355068012 +0000 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-27 06:38:14.355068012 +0000 2024-05-27T06:46:09.580 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-27 06:38:14.355068012 +0000 2024-05-27T06:46:09.580 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T06:46:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-27T06:46:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-27T06:46:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000375602 s, 1.4 MB/s 2024-05-27T06:46:09.663 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T06:46:09.718 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-27 06:45:41.757062675 +0000 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-27 06:38:14.953058444 +0000 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-27 06:38:14.953058444 +0000 2024-05-27T06:46:09.768 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-27 06:38:14.953058444 +0000 2024-05-27T06:46:09.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T06:46:09.834 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-27T06:46:09.834 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-27T06:46:09.834 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000361583 s, 1.4 MB/s 2024-05-27T06:46:09.837 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T06:46:09.882 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-27 06:45:41.757062675 +0000 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-27 06:38:15.558048764 +0000 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-27 06:38:15.558048764 +0000 2024-05-27T06:46:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-27 06:38:15.558048764 +0000 2024-05-27T06:46:09.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T06:46:10.074 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-27T06:46:10.074 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-27T06:46:10.074 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000351691 s, 1.5 MB/s 2024-05-27T06:46:10.077 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T06:46:10.118 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-27 06:45:41.758062659 +0000 2024-05-27T06:46:10.211 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-27 06:38:16.138039485 +0000 2024-05-27T06:46:10.212 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-27 06:38:16.138039485 +0000 2024-05-27T06:46:10.212 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-27 06:38:16.138039485 +0000 2024-05-27T06:46:10.212 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T06:46:10.278 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-27T06:46:10.278 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-27T06:46:10.278 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000350569 s, 1.5 MB/s 2024-05-27T06:46:10.281 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T06:46:10.326 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-05-27T06:46:10.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T06:46:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:46:11.009 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.009 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.010 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:10 smithi164 ceph-mon[35327]: Deploying daemon mgr.z on smithi164 2024-05-27T06:46:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:10 smithi016 ceph-mon[31415]: Deploying daemon mgr.z on smithi164 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:11.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:11.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:11.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:11.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:10 smithi069 ceph-mon[35276]: Deploying daemon mgr.z on smithi164 2024-05-27T06:46:12.833 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:12 smithi164 systemd[1]: Starting Ceph mgr.z for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:46:13.200 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:12 smithi164 ceph-mon[35327]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:13.202 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:12 smithi164 podman[37149]: 2024-05-27 06:46:12.836067416 +0000 UTC m=+0.033348715 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:46:13.202 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:12 smithi164 podman[37149]: 2024-05-27 06:46:12.986939024 +0000 UTC m=+0.184220350 container create 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T06:46:13.202 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 podman[37149]: 2024-05-27 06:46:13.207594579 +0000 UTC m=+0.404875915 container init 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-27T06:46:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:12 smithi016 ceph-mon[31415]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:13.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:12 smithi069 ceph-mon[35276]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:13.528 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 podman[37149]: 2024-05-27 06:46:13.220853449 +0000 UTC m=+0.418134778 container start 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T06:46:13.528 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 ceph-mgr[37168]: -- 172.21.15.164:0/2122695914 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef212e85a0 con 0x55ef26072480 2024-05-27T06:46:13.528 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 bash[37149]: 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 2024-05-27T06:46:13.528 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 systemd[1]: Started Ceph mgr.z for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:46:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T06:46:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T06:46:13.660 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-27T06:46:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-27T06:46:13.661 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0401582 s, 261 MB/s 2024-05-27T06:46:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-27T06:46:13.788 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:13.532+0000 7fb72fe7e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T06:46:13.788 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:13.793+0000 7fb72fe7e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:46:14.083 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:13.927+0000 7fb72fe7e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T06:46:14.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:14.334+0000 7fb72fe7e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:14.449+0000 7fb72fe7e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: Deploying daemon mgr.y on smithi069 2024-05-27T06:46:14.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:14 smithi164 ceph-mon[35327]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: Deploying daemon mgr.y on smithi069 2024-05-27T06:46:14.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:14 smithi069 ceph-mon[35276]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T06:46:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: Deploying daemon mgr.y on smithi069 2024-05-27T06:46:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:14 smithi016 ceph-mon[31415]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:46:16.333 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:16.058+0000 7fb72fe7e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T06:46:16.333 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:16.288+0000 7fb72fe7e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T06:46:16.621 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 systemd[1]: Starting Ceph mgr.y for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:46:16.833 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:16.507+0000 7fb72fe7e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T06:46:16.908 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 podman[36625]: 2024-05-27 06:46:16.627587978 +0000 UTC m=+0.126016310 container create c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T06:46:16.908 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 podman[36625]: 2024-05-27 06:46:16.53336117 +0000 UTC m=+0.031789477 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:46:16.908 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 podman[36625]: 2024-05-27 06:46:16.914428627 +0000 UTC m=+0.412857016 container init c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T06:46:17.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:16 smithi016 ceph-mon[31415]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 -- 172.21.15.16:0/299103185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58104360 msgr2=0x7f7e58104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 --2- 172.21.15.16:0/299103185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58104360 0x7f7e58104770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009990 tx=0x7f7e44009ca0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 -- 172.21.15.16:0/299103185 shutdown_connections 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 --2- 172.21.15.16:0/299103185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104d00 0x7f7e5810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 --2- 172.21.15.16:0/299103185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58103230 0x7f7e58103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 --2- 172.21.15.16:0/299103185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58104360 0x7f7e58104770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 -- 172.21.15.16:0/299103185 >> 172.21.15.16:0/299103185 conn(0x7f7e580fe870 msgr2=0x7f7e58100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 -- 172.21.15.16:0/299103185 shutdown_connections 2024-05-27T06:46:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.211+0000 7f7e5f400700 1 -- 172.21.15.16:0/299103185 wait complete. 2024-05-27T06:46:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.212+0000 7f7e5f400700 1 Processor -- start 2024-05-27T06:46:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.212+0000 7f7e5f400700 1 -- start start 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104360 0x7f7e5819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58104d00 0x7f7e581a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810eb90 con 0x7f7e58103230 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810ea10 con 0x7f7e58104360 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5f400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810ed10 con 0x7f7e58104d00 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45576/0 (socket says 172.21.15.16:45576) 2024-05-27T06:46:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5ca00700 1 -- 172.21.15.16:0/3538316227 learned_addr learned my addr 172.21.15.16:0/3538316227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:46:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e57e00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104360 0x7f7e5819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.213+0000 7f7e5d400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58104d00 0x7f7e581a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 -- 172.21.15.16:0/3538316227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58104d00 msgr2=0x7f7e581a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58104d00 0x7f7e581a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 -- 172.21.15.16:0/3538316227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104360 msgr2=0x7f7e5819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104360 0x7f7e5819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 -- 172.21.15.16:0/3538316227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e581a4630 con 0x7f7e58103230 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.214+0000 7f7e5ca00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00d950 tx=0x7f7e4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.215+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e58103230 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.215+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e4c00f690 con 0x7f7e58103230 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.215+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0039e0 con 0x7f7e58103230 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.215+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e581a4920 con 0x7f7e58103230 2024-05-27T06:46:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.215+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e581a4e10 con 0x7f7e58103230 2024-05-27T06:46:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.216+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e580512b0 con 0x7f7e58103230 2024-05-27T06:46:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.216+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7e4c003b80 con 0x7f7e58103230 2024-05-27T06:46:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.217+0000 7f7e55600700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 0x7f7e40040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.217+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e4c052440 con 0x7f7e58103230 2024-05-27T06:46:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.217+0000 7f7e57e00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 0x7f7e40040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.218+0000 7f7e57e00700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 0x7f7e40040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e5819db80 tx=0x7f7e4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.224+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e4c020030 con 0x7f7e58103230 2024-05-27T06:46:17.231 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:16 smithi069 ceph-mon[35276]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:17.232 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 podman[36625]: 2024-05-27 06:46:16.926932249 +0000 UTC m=+0.425360585 container start c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T06:46:17.232 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:16 smithi069 bash[36625]: c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca 2024-05-27T06:46:17.232 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:17 smithi069 systemd[1]: Started Ceph mgr.y for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:46:17.232 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:17 smithi069 ceph-mgr[36644]: -- 172.21.15.69:0/2346624708 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617e30be5a0 con 0x5617e7e52480 2024-05-27T06:46:17.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:16 smithi164 ceph-mon[35327]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:17.270 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:16.891+0000 7fb72fe7e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T06:46:17.271 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:17.015+0000 7fb72fe7e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T06:46:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:17.476+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7e58100960 con 0x7f7e4003e750 2024-05-27T06:46:17.485 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:17.237+0000 7f36c5160200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T06:46:17.486 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:17.492+0000 7f36c5160200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:46:17.531 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:17.276+0000 7fb72fe7e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T06:46:17.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:17.538+0000 7fb72fe7e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:46:17.869 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:17.623+0000 7f36c5160200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T06:46:18.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:18.041+0000 7f36c5160200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T06:46:18.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:18.158+0000 7f36c5160200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:18.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:18.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.582 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:18.383+0000 7fb72fe7e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:18.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:19.007 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:18.685+0000 7fb72fe7e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T06:46:19.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:19 smithi164 ceph-mon[35327]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:19.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:19 smithi164 ceph-mon[35327]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:19.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:19 smithi016 ceph-mon[31415]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:19.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:19 smithi016 ceph-mon[31415]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:19.583 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:19.288+0000 7fb72fe7e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T06:46:19.583 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:19.441+0000 7fb72fe7e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T06:46:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:19 smithi069 ceph-mon[35276]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:19 smithi069 ceph-mon[35276]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:20.120 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:19.808+0000 7f36c5160200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T06:46:20.120 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:20.046+0000 7f36c5160200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T06:46:20.333 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:19.909+0000 7fb72fe7e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T06:46:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:20 smithi016 ceph-mon[31415]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:20 smithi069 ceph-mon[35276]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:20.620 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:20.271+0000 7f36c5160200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T06:46:20.798 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:20.360+0000 7fb72fe7e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T06:46:20.798 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:20.485+0000 7fb72fe7e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:20.805+0000 7fb72fe7e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:20.799 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:20 smithi164 ceph-mon[35327]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:21.039 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:20.663+0000 7f36c5160200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T06:46:21.039 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:20.785+0000 7f36c5160200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T06:46:21.082 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:20.987+0000 7fb72fe7e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T06:46:21.295 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:21.044+0000 7f36c5160200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T06:46:21.440 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:21.126+0000 7fb72fe7e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T06:46:21.619 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:21.301+0000 7f36c5160200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:21 smithi069 ceph-mon[35276]: Reconfiguring daemon mgr.x on smithi016 2024-05-27T06:46:21.757 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:21.548+0000 7fb72fe7e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T06:46:21.757 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:21.678+0000 7fb72fe7e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.758 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:21 smithi164 ceph-mon[35327]: Reconfiguring daemon mgr.x on smithi016 2024-05-27T06:46:21.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:21 smithi016 ceph-mon[31415]: Reconfiguring daemon mgr.x on smithi016 2024-05-27T06:46:22.370 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:22.088+0000 7f36c5160200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T06:46:22.768 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:22.479+0000 7fb72fe7e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T06:46:22.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:22 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]: dispatch 2024-05-27T06:46:22.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:22 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]': finished 2024-05-27T06:46:22.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:22 smithi164 ceph-mon[35327]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T06:46:22.769 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:22.769 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:22 smithi164 ceph-mon[35327]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:22.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:22 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]: dispatch 2024-05-27T06:46:22.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:22 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]': finished 2024-05-27T06:46:22.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:22 smithi016 ceph-mon[31415]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T06:46:22.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:22.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:22 smithi016 ceph-mon[31415]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:22.869 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:22.376+0000 7f36c5160200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T06:46:22.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:22 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]: dispatch 2024-05-27T06:46:22.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:22 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2276215067' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b"}]': finished 2024-05-27T06:46:22.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:22 smithi069 ceph-mon[35276]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T06:46:22.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:22.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:22 smithi069 ceph-mon[35276]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:23.083 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 06:46:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T06:46:22.775+0000 7fb72fe7e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T06:46:23.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:22.927+0000 7f36c5160200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T06:46:23.370 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:23.054+0000 7f36c5160200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T06:46:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:23.574+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f7e4c004220 con 0x7f7e58103230 2024-05-27T06:46:23.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: Standby manager daemon z started 2024-05-27T06:46:23.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T06:46:23.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:23.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T06:46:23.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:23.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:23 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3326190407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:46:23.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: Standby manager daemon z started 2024-05-27T06:46:23.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T06:46:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T06:46:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:23 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3326190407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:46:23.869 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:23.514+0000 7f36c5160200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T06:46:23.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: Standby manager daemon z started 2024-05-27T06:46:23.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T06:46:23.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:23.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T06:46:23.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/1473110883' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:23.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:23 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3326190407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:46:24.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:23.964+0000 7f36c5160200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T06:46:24.370 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:24.087+0000 7f36c5160200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T06:46:24.711 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:24 smithi069 ceph-mon[35276]: mgrmap e14: x(active, since 105s), standbys: z 2024-05-27T06:46:24.711 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T06:46:24.712 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:24 smithi069 ceph-mon[35276]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:24.712 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:24.407+0000 7f36c5160200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T06:46:24.712 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:24.589+0000 7f36c5160200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T06:46:24.712 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:24.717+0000 7f36c5160200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T06:46:24.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:24 smithi164 ceph-mon[35327]: mgrmap e14: x(active, since 105s), standbys: z 2024-05-27T06:46:24.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T06:46:24.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:24 smithi164 ceph-mon[35327]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:24.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:24 smithi016 ceph-mon[31415]: mgrmap e14: x(active, since 105s), standbys: z 2024-05-27T06:46:24.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T06:46:24.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:24 smithi016 ceph-mon[31415]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:25.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:25.116+0000 7f36c5160200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T06:46:25.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:25.241+0000 7f36c5160200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T06:46:26.297 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:26.014+0000 7f36c5160200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T06:46:26.297 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 06:46:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T06:46:26.303+0000 7f36c5160200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T06:46:27.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:27.012+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f7e4c00ec90 con 0x7f7e58103230 2024-05-27T06:46:27.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:27.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: Standby manager daemon y started 2024-05-27T06:46:27.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T06:46:27.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:27.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T06:46:27.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:26 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: Standby manager daemon y started 2024-05-27T06:46:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T06:46:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:27.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T06:46:27.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:26 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: Standby manager daemon y started 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T06:46:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:26 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/999775945' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T06:46:28.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:28 smithi016 ceph-mon[31415]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-05-27T06:46:28.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:28 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T06:46:28.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:28 smithi164 ceph-mon[35327]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-05-27T06:46:28.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:28 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T06:46:28.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:28 smithi069 ceph-mon[35276]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-05-27T06:46:28.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:28 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T06:46:29.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:29 smithi016 ceph-mon[31415]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:29.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:29.272 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:29 smithi069 ceph-mon[35276]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:29.273 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.273 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.273 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:29.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:29 smithi164 ceph-mon[35327]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:29.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:29.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:30.179 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:30.179 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:30.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:30.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:30.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:30.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:31.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:31 smithi069 ceph-mon[35276]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:31.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:31.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:31 smithi164 ceph-mon[35327]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:31.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:31 smithi016 ceph-mon[31415]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:32.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T06:46:32.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:32.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T06:46:32.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:32.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T06:46:32.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:33.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:33 smithi016 ceph-mon[31415]: Deploying daemon osd.0 on smithi016 2024-05-27T06:46:33.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:33 smithi016 ceph-mon[31415]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:33.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:33 smithi164 ceph-mon[35327]: Deploying daemon osd.0 on smithi016 2024-05-27T06:46:33.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:33 smithi164 ceph-mon[35327]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:33.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:33 smithi069 ceph-mon[35276]: Deploying daemon osd.0 on smithi016 2024-05-27T06:46:33.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:33 smithi069 ceph-mon[35276]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:35.293 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:35 smithi016 ceph-mon[31415]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:35.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:35 smithi164 ceph-mon[35327]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:35.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:35 smithi069 ceph-mon[35276]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:37 smithi164 ceph-mon[35327]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:37 smithi016 ceph-mon[31415]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:37.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:37 smithi069 ceph-mon[35276]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:39.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:39 smithi164 ceph-mon[35327]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:39 smithi016 ceph-mon[31415]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:39 smithi069 ceph-mon[35276]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:41.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:41 smithi016 ceph-mon[31415]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:41 smithi164 ceph-mon[35327]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:41 smithi069 ceph-mon[35276]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:42.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:42 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:42 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:42 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:42.586 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:42 smithi164 ceph-mon[35327]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:42.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:42 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:42.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:42 smithi016 ceph-mon[31415]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:42.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:42 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:42 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:42.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:42 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:42.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:42 smithi069 ceph-mon[35276]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:45.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:45.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:46.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:45 smithi016 ceph-mon[31415]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T06:46:46.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:45 smithi164 ceph-mon[35327]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T06:46:46.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:45 smithi069 ceph-mon[35276]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T06:46:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.095+0000 7f7e55600700 1 -- 172.21.15.16:0/3538316227 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7e58100960 con 0x7f7e4003e750 2024-05-27T06:46:47.090 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 msgr2=0x7f7e40040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 0x7f7e40040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e5819db80 tx=0x7f7e4401a040 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 msgr2=0x7f7e5819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00d950 tx=0x7f7e4c00dc60 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 shutdown_connections 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7e4003e750 0x7f7e40040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e58104d00 0x7f7e581a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e58103230 0x7f7e5819cad0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 --2- 172.21.15.16:0/3538316227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58104360 0x7f7e5819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.099+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 >> 172.21.15.16:0/3538316227 conn(0x7f7e580fe870 msgr2=0x7f7e58100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.100+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 shutdown_connections 2024-05-27T06:46:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:46:47.100+0000 7f7e5f400700 1 -- 172.21.15.16:0/3538316227 wait complete. 2024-05-27T06:46:47.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:47 smithi164 ceph-mon[35327]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:47.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:47 smithi164 ceph-mon[35327]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T06:46:47.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:47 smithi164 ceph-mon[35327]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T06:46:47.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:47 smithi164 ceph-mon[35327]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:46:47.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:47 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:47 smithi016 ceph-mon[31415]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:47 smithi016 ceph-mon[31415]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T06:46:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:47 smithi016 ceph-mon[31415]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T06:46:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:47 smithi016 ceph-mon[31415]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:46:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:47 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:47.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:47 smithi069 ceph-mon[35276]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:47.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:47 smithi069 ceph-mon[35276]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T06:46:47.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:47 smithi069 ceph-mon[35276]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T06:46:47.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:47 smithi069 ceph-mon[35276]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:46:47.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:47 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.100 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T06:46:48.107 INFO:tasks.cephadm:Deploying osd.1 on smithi069 with /dev/vg_nvme/lv_4... 2024-05-27T06:46:48.107 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T06:46:48.284 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:46:48.284 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:48.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:48 smithi016 ceph-mon[31415]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:48.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:48 smithi164 ceph-mon[35327]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:46:48.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:48 smithi069 ceph-mon[35276]: from='osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191]' entity='osd.0' 2024-05-27T06:46:48.544 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] boot 2024-05-27T06:46:49.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T06:46:49.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] boot 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T06:46:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:49.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: osd.0 [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] boot 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T06:46:49.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:51 smithi164 ceph-mon[35327]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:51 smithi164 ceph-mon[35327]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T06:46:51.342 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:51 smithi069 ceph-mon[35276]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:51.342 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:51 smithi069 ceph-mon[35276]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367095 s, 286 MB/s 2024-05-27T06:46:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-27T06:46:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:51 smithi016 ceph-mon[31415]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:51 smithi016 ceph-mon[31415]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T06:46:52.084 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-05-27T06:46:52.495 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:46:53.332 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:53 smithi069 ceph-mon[35276]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:53 smithi164 ceph-mon[35327]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:53.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:53 smithi016 ceph-mon[31415]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi016 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 17094M 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:54.322 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 -- 172.21.15.69:0/4072393973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec103ed0 msgr2=0x7ff8ec1042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4072393973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec103ed0 0x7ff8ec1042e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400b0a0 tx=0x7ff8e400b3b0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 -- 172.21.15.69:0/4072393973 shutdown_connections 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4072393973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec103ed0 0x7ff8ec1042e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4072393973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec102c30 0x7ff8ec103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4072393973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec103580 0x7ff8ec103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.572+0000 7ff8f2000700 1 -- 172.21.15.69:0/4072393973 >> 172.21.15.69:0/4072393973 conn(0x7ff8ec0fe860 msgr2=0x7ff8ec100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.573+0000 7ff8f2000700 1 -- 172.21.15.69:0/4072393973 shutdown_connections 2024-05-27T06:46:54.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.573+0000 7ff8f2000700 1 -- 172.21.15.69:0/4072393973 wait complete. 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 Processor -- start 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 -- start start 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec103580 0x7ff8ec19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec103ed0 0x7ff8ec1a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111ea0 con 0x7ff8ec103580 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111d20 con 0x7ff8ec102c30 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8f2000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec112020 con 0x7ff8ec103ed0 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8eb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8eb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40880/0 (socket says 172.21.15.69:40880) 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.574+0000 7ff8eb400700 1 -- 172.21.15.69:0/4260803606 learned_addr learned my addr 172.21.15.69:0/4260803606 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eaa00700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec103580 0x7ff8ec19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 -- 172.21.15.69:0/4260803606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec103ed0 msgr2=0x7ff8ec1a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec103ed0 0x7ff8ec1a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 -- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec103580 msgr2=0x7ff8ec19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec103580 0x7ff8ec19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 -- 172.21.15.69:0/4260803606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec1a4540 con 0x7ff8ec102c30 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8eb400700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00d980 tx=0x7ff8dc00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.575+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc017070 con 0x7ff8ec102c30 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.576+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8dc010460 con 0x7ff8ec102c30 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.576+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec1a4830 con 0x7ff8ec102c30 2024-05-27T06:46:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.576+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00de60 con 0x7ff8ec102c30 2024-05-27T06:46:54.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.576+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a4dc0 con 0x7ff8ec102c30 2024-05-27T06:46:54.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.577+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec0512b0 con 0x7ff8ec102c30 2024-05-27T06:46:54.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.580+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff8dc010610 con 0x7ff8ec102c30 2024-05-27T06:46:54.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.581+0000 7ff8e3e00700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 0x7ff8d00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:46:54.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.581+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff8dc0e0290 con 0x7ff8ec102c30 2024-05-27T06:46:54.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.582+0000 7ff8eaa00700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 0x7ff8d00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:46:54.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.583+0000 7ff8eaa00700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 0x7ff8d00b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005950 tx=0x7ff8d8005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:46:54.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.587+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8dc0a9a00 con 0x7ff8ec102c30 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi016 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 17094M 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:54.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi016 2024-05-27T06:46:54.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 17094M 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:46:54.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:46:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:46:54.828+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff8ec063cb0 con 0x7ff8d00b2870 2024-05-27T06:46:55.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:55 smithi164 ceph-mon[35327]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:55.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:55.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:55.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:55 smithi016 ceph-mon[31415]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:55.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:55 smithi069 ceph-mon[35276]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:46:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:46:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:46:56.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:56 smithi069 ceph-mon[35276]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:56.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:56 smithi164 ceph-mon[35327]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:56 smithi016 ceph-mon[31415]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:46:57.429 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:57 smithi069 ceph-mon[35276]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:57.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:57 smithi164 ceph-mon[35327]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:57.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:57 smithi016 ceph-mon[31415]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/2871615657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]': finished 2024-05-27T06:46:59.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T06:46:59.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:46:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:46:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/2871615657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]': finished 2024-05-27T06:46:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T06:46:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:46:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/2871615657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]: dispatch 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f0fa1e6-b4a7-48a6-b89d-c5612a23e676"}]': finished 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T06:46:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:46:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:00.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:00 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/1856814941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:00 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/1856814941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:00 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/1856814941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:01.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:01 smithi164 ceph-mon[35327]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:01 smithi016 ceph-mon[31415]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:01.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:01 smithi069 ceph-mon[35276]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:03 smithi164 ceph-mon[35327]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:03.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:03 smithi016 ceph-mon[31415]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:03.622 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:03 smithi069 ceph-mon[35276]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:05.353 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:05 smithi069 ceph-mon[35276]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:05.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:05 smithi164 ceph-mon[35327]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:05 smithi016 ceph-mon[31415]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:07.509 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:07 smithi069 ceph-mon[35276]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:07 smithi164 ceph-mon[35327]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:07.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:07 smithi016 ceph-mon[31415]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:08.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T06:47:08.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:08.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T06:47:08.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T06:47:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:09 smithi164 ceph-mon[35327]: Deploying daemon osd.1 on smithi069 2024-05-27T06:47:09.585 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:09 smithi164 ceph-mon[35327]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:09 smithi016 ceph-mon[31415]: Deploying daemon osd.1 on smithi069 2024-05-27T06:47:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:09 smithi016 ceph-mon[31415]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:09.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:09 smithi069 ceph-mon[35276]: Deploying daemon osd.1 on smithi069 2024-05-27T06:47:09.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:09 smithi069 ceph-mon[35276]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:11.366 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:11 smithi069 ceph-mon[35276]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:11.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:11 smithi164 ceph-mon[35327]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:11 smithi016 ceph-mon[31415]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:12.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:12 smithi164 ceph-mon[35327]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:12.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:12 smithi016 ceph-mon[31415]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:12 smithi069 ceph-mon[35276]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:14 smithi164 ceph-mon[35327]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:14 smithi016 ceph-mon[31415]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:15.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:14 smithi069 ceph-mon[35276]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:17.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:16 smithi164 ceph-mon[35327]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:17.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:16 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:17.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:16 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:17.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:16 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:17.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:16 smithi016 ceph-mon[31415]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:17.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:16 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:16 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:16 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:17.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:16 smithi069 ceph-mon[35276]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:17.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:16 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:17.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:16 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:17.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:16 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:19.289 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:19 smithi069 ceph-mon[35276]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:19.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:18 smithi164 ceph-mon[35327]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:19.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:18 smithi016 ceph-mon[31415]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:20 smithi164 ceph-mon[35327]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:21.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:21.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:21.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:20 smithi016 ceph-mon[31415]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:21.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:21.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:21.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:21.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:20 smithi069 ceph-mon[35276]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:21.882 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 1 on host 'smithi069' 2024-05-27T06:47:21.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.889+0000 7ff8e3e00700 1 -- 172.21.15.69:0/4260803606 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8ec063cb0 con 0x7ff8d00b2870 2024-05-27T06:47:21.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.892+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 msgr2=0x7ff8d00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:21.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.892+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 0x7ff8d00b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005950 tx=0x7ff8d8005ed0 comp rx=0 tx=0).stop 2024-05-27T06:47:21.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 msgr2=0x7ff8ec19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00d980 tx=0x7ff8dc00dd40 comp rx=0 tx=0).stop 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 shutdown_connections 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8d00b2870 0x7ff8d00b4d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec103ed0 0x7ff8ec1a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ec102c30 0x7ff8ec19c9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 --2- 172.21.15.69:0/4260803606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec103580 0x7ff8ec19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 >> 172.21.15.69:0/4260803606 conn(0x7ff8ec0fe860 msgr2=0x7ff8ec10b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 shutdown_connections 2024-05-27T06:47:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:47:21.893+0000 7ff8f2000700 1 -- 172.21.15.69:0/4260803606 wait complete. 2024-05-27T06:47:22.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:21 smithi016 ceph-mon[31415]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:21 smithi016 ceph-mon[31415]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:22.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:21 smithi069 ceph-mon[35276]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:21 smithi069 ceph-mon[35276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:22.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:21 smithi164 ceph-mon[35327]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:21 smithi164 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T06:47:22.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:22.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:22.619 DEBUG:teuthology.orchestra.run.smithi069:osd.1> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T06:47:22.626 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/vg_nvme/lv_4... 2024-05-27T06:47:22.626 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T06:47:23.074 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:47:23.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:22 smithi164 ceph-mon[35327]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:23.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T06:47:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T06:47:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:22 smithi016 ceph-mon[31415]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:23.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T06:47:23.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T06:47:23.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: from='osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:47:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:22 smithi069 ceph-mon[35276]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:24.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:23 smithi069 ceph-mon[35276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:47:24.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:23 smithi069 ceph-mon[35276]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T06:47:24.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:23 smithi069 ceph-mon[35276]: from='osd.1 ' entity='osd.1' 2024-05-27T06:47:24.120 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:47:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1[39824]: 2024-05-27T06:47:23.857+0000 7f1d3d600700 -1 osd.1 0 waiting for initial osdmap 2024-05-27T06:47:24.121 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:47:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1[39824]: 2024-05-27T06:47:23.877+0000 7f1d33400700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T06:47:24.210 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:23 smithi164 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:47:24.210 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:23 smithi164 ceph-mon[35327]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T06:47:24.211 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.211 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.211 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:23 smithi164 ceph-mon[35327]: from='osd.1 ' entity='osd.1' 2024-05-27T06:47:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:23 smithi016 ceph-mon[31415]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:47:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:23 smithi016 ceph-mon[31415]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T06:47:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:23 smithi016 ceph-mon[31415]: from='osd.1 ' entity='osd.1' 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] boot 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] boot 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T06:47:25.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: osd.1 [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] boot 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T06:47:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T06:47:25.717 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T06:47:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T06:47:25.718 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-27T06:47:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-27T06:47:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0338659 s, 310 MB/s 2024-05-27T06:47:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-27T06:47:26.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-05-27T06:47:26.919 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:47:27.213 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:26 smithi164 ceph-mon[35327]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T06:47:27.213 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:26 smithi164 ceph-mon[35327]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:26 smithi016 ceph-mon[31415]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T06:47:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:26 smithi016 ceph-mon[31415]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:26 smithi069 ceph-mon[35276]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T06:47:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:26 smithi069 ceph-mon[35276]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 -- 172.21.15.164:0/1766495740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 msgr2=0x7f74f0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 --2- 172.21.15.164:0/1766495740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f0104770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009990 tx=0x7f74dc009ca0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 -- 172.21.15.164:0/1766495740 shutdown_connections 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 --2- 172.21.15.164:0/1766495740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 --2- 172.21.15.164:0/1766495740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 0x7f74f0103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 --2- 172.21.15.164:0/1766495740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f0104770 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.084+0000 7f74f5200700 1 -- 172.21.15.164:0/1766495740 >> 172.21.15.164:0/1766495740 conn(0x7f74f00fe870 msgr2=0x7f74f0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.085+0000 7f74f5200700 1 -- 172.21.15.164:0/1766495740 shutdown_connections 2024-05-27T06:47:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.085+0000 7f74f5200700 1 -- 172.21.15.164:0/1766495740 wait complete. 2024-05-27T06:47:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 Processor -- start 2024-05-27T06:47:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 -- start start 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 0x7f74f019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f01a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010eb90 con 0x7f74f0103230 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010ea10 con 0x7f74f0104d00 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.086+0000 7f74f5200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010ed10 con 0x7f74f0104360 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55306/0 (socket says 172.21.15.164:55306) 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 -- 172.21.15.164:0/430310919 learned_addr learned my addr 172.21.15.164:0/430310919 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ede00700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 0x7f74f019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ee800700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f01a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 -- 172.21.15.164:0/430310919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 msgr2=0x7f74f01a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:29.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f01a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 -- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 msgr2=0x7f74f019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 0x7f74f019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ed400700 1 -- 172.21.15.164:0/430310919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f01a4640 con 0x7f74f0104360 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.087+0000 7f74ee800700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f01a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74ed400700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:47:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc003c10 con 0x7f74f0104360 2024-05-27T06:47:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc004830 con 0x7f74f0104360 2024-05-27T06:47:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc005350 con 0x7f74f0104360 2024-05-27T06:47:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01a48d0 con 0x7f74f0104360 2024-05-27T06:47:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.088+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a4c70 con 0x7f74f0104360 2024-05-27T06:47:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.090+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f00512b0 con 0x7f74f0104360 2024-05-27T06:47:29.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.093+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f74dc0054f0 con 0x7f74f0104360 2024-05-27T06:47:29.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.094+0000 7f74e6a00700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 0x7f74d40b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:47:29.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.095+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f74dc0e1a80 con 0x7f74f0104360 2024-05-27T06:47:29.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.095+0000 7f74ede00700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 0x7f74d40b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:47:29.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.096+0000 7f74ede00700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 0x7f74d40b4e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:47:29.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.101+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74dc013070 con 0x7f74f0104360 2024-05-27T06:47:29.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:29.344+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f74f0100970 con 0x7f74d40b28f0 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi069 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:47:29.337 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.338 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:29.338 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:29.338 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi069 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.561 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:29.562 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:29.562 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi069 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:30 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 17089M 2024-05-27T06:47:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:47:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:47:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:30.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:30 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 17089M 2024-05-27T06:47:30.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:47:30.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:47:30.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:30.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:30 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 17089M 2024-05-27T06:47:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:47:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:47:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:31 smithi164 ceph-mon[35327]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:47:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:31 smithi164 ceph-mon[35327]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:31 smithi016 ceph-mon[31415]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:47:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:31 smithi016 ceph-mon[31415]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:31 smithi069 ceph-mon[35276]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:47:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:31 smithi069 ceph-mon[35276]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:32.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:32 smithi164 ceph-mon[35327]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:32 smithi016 ceph-mon[31415]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:32 smithi069 ceph-mon[35276]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:33.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:33 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/1305944519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:33 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:33 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]': finished 2024-05-27T06:47:33.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:33 smithi164 ceph-mon[35327]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T06:47:33.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:33.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:33 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/1305944519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:33 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:33 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]': finished 2024-05-27T06:47:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:33 smithi016 ceph-mon[31415]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T06:47:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:33.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:33 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/1305944519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:33 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]: dispatch 2024-05-27T06:47:33.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:33 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebccb12e-a7d2-4907-9140-78f8ab1994bc"}]': finished 2024-05-27T06:47:33.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:33 smithi069 ceph-mon[35276]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T06:47:33.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:34.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:34 smithi164 ceph-mon[35327]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:34.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:34 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/2748470043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:34.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:34 smithi016 ceph-mon[31415]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:34.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:34 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/2748470043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:34.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:34 smithi069 ceph-mon[35276]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:34.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:34 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/2748470043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:47:37.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:36 smithi164 ceph-mon[35327]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:36 smithi016 ceph-mon[31415]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:37.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:36 smithi069 ceph-mon[35276]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:39.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:38 smithi164 ceph-mon[35327]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:38 smithi016 ceph-mon[31415]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:38 smithi069 ceph-mon[35276]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:41.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:41 smithi164 ceph-mon[35327]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:41.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:41 smithi016 ceph-mon[31415]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:41 smithi069 ceph-mon[35276]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:43.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:43 smithi164 ceph-mon[35327]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:43.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T06:47:43.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:43 smithi016 ceph-mon[31415]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:43 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T06:47:43.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:43 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:43 smithi069 ceph-mon[35276]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:43 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T06:47:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:43 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:44.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:44 smithi164 ceph-mon[35327]: Deploying daemon osd.2 on smithi164 2024-05-27T06:47:44.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:44 smithi016 ceph-mon[31415]: Deploying daemon osd.2 on smithi164 2024-05-27T06:47:44.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:44 smithi069 ceph-mon[35276]: Deploying daemon osd.2 on smithi164 2024-05-27T06:47:45.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:45 smithi164 ceph-mon[35327]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:45 smithi016 ceph-mon[31415]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:45.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:45 smithi069 ceph-mon[35276]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:47.223 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:47 smithi164 ceph-mon[35327]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:47 smithi016 ceph-mon[31415]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:47.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:47 smithi069 ceph-mon[35276]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:49.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:49 smithi164 ceph-mon[35327]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:49 smithi016 ceph-mon[31415]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:49.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:49 smithi069 ceph-mon[35276]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:51.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:51 smithi164 ceph-mon[35327]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:51.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:51 smithi016 ceph-mon[31415]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:51.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:51 smithi069 ceph-mon[35276]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:52.718 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:52 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.718 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:52 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.718 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:52 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:52.718 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:52 smithi164 ceph-mon[35327]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:52 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:52 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:52 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:52 smithi016 ceph-mon[31415]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:52.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:52 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:52 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:52.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:52 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:52.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:52 smithi069 ceph-mon[35276]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:55.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:54 smithi164 ceph-mon[35327]: pgmap v103: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:55.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:55.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:55.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:54 smithi016 ceph-mon[31415]: pgmap v103: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:55.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:55.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:55.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:54 smithi069 ceph-mon[35276]: pgmap v103: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:55.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:55.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:56.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:56 smithi164 ceph-mon[35327]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:56 smithi164 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:56.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:56.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:56.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:56 smithi016 ceph-mon[31415]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:56 smithi016 ceph-mon[31415]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:56.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:56.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:56.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:56 smithi069 ceph-mon[35276]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:56 smithi069 ceph-mon[35276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T06:47:56.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:47:56.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:47:56.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:57.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.120+0000 7f74e6a00700 1 -- 172.21.15.164:0/430310919 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74f0100970 con 0x7f74d40b28f0 2024-05-27T06:47:57.116 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2024-05-27T06:47:57.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 msgr2=0x7f74d40b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:57.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 0x7f74d40b4e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 msgr2=0x7f74f019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0047b0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 shutdown_connections 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f74d40b28f0 0x7f74d40b4e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f0104d00 0x7f74f01a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f0103230 0x7f74f019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 --2- 172.21.15.164:0/430310919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0104360 0x7f74f019d020 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 >> 172.21.15.164:0/430310919 conn(0x7f74f00fe870 msgr2=0x7f74f0100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 shutdown_connections 2024-05-27T06:47:57.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:47:57.124+0000 7f74f5200700 1 -- 172.21.15.164:0/430310919 wait complete. 2024-05-27T06:47:57.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:57 smithi164 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:57.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T06:47:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T06:47:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:57 smithi016 ceph-mon[31415]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:57.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T06:47:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T06:47:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: from='osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:57 smithi069 ceph-mon[35276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:47:57.958 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T06:47:57.964 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-05-27T06:47:57.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T06:47:58.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:47:58.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T06:47:58.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:47:58.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:47:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: pgmap v107: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] boot 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T06:47:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:59.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:47:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:47:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: pgmap v107: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] boot 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:47:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:47:59.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: pgmap v107: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: osd.2 [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] boot 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T06:47:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:47:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:48:00.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:00.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 ceph-mon[31415]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T06:48:00.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:48:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 ceph-mon[35327]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T06:48:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:48:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:00.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:00 smithi069 ceph-mon[35276]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T06:48:00.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47972]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47972]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47972]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47975]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47975]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47975]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.500 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47975]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43545]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.870 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 06:48:00 smithi069 sudo[43548]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42036]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42036]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42036]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42036]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42042]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:00.895 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42042]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:00.896 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42042]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47984]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47984]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47987]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:00 smithi016 sudo[47987]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42058]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42058]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42058]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42058]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42063]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:00 smithi164 sudo[42063]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:48:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T06:48:01.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43551]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43551]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43554]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43554]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 sudo[43554]: pam_unix(sudo:session): session closed for user root 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T06:48:01.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0381097 s, 275 MB/s 2024-05-27T06:48:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-27T06:48:02.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:02 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:02.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:02 smithi164 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:02.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:02 smithi164 ceph-mon[35327]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T06:48:02.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:02 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:02.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:02 smithi016 ceph-mon[31415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:02.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:02 smithi016 ceph-mon[31415]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T06:48:02.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:02 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T06:48:02.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:02 smithi069 ceph-mon[35276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T06:48:02.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:02 smithi069 ceph-mon[35276]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T06:48:02.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-05-27T06:48:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:48:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:48:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: Cluster is now healthy 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: Cluster is now healthy 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:03.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:03.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:48:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: Cluster is now healthy 2024-05-27T06:48:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:03.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:04.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:04 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi164 2024-05-27T06:48:04.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:04 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 17089M 2024-05-27T06:48:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:04 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi164 2024-05-27T06:48:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:04 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 17089M 2024-05-27T06:48:04.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:04 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi164 2024-05-27T06:48:04.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:04 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 17089M 2024-05-27T06:48:05.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:05 smithi016 ceph-mon[31415]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:05.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:05 smithi069 ceph-mon[35276]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:05.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:05 smithi164 ceph-mon[35327]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 -- 172.21.15.16:0/3864968615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 msgr2=0x7f2a3410e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 --2- 172.21.15.16:0/3864968615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a3410e8c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00b0a0 tx=0x7f2a2c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 -- 172.21.15.16:0/3864968615 shutdown_connections 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 --2- 172.21.15.16:0/3864968615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a3410e8c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 --2- 172.21.15.16:0/3864968615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34106540 0x7f2a34106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 --2- 172.21.15.16:0/3864968615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34107670 0x7f2a34107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.874+0000 7f2a3a200700 1 -- 172.21.15.16:0/3864968615 >> 172.21.15.16:0/3864968615 conn(0x7f2a34101b80 msgr2=0x7f2a34104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.875+0000 7f2a3a200700 1 -- 172.21.15.16:0/3864968615 shutdown_connections 2024-05-27T06:48:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.875+0000 7f2a3a200700 1 -- 172.21.15.16:0/3864968615 wait complete. 2024-05-27T06:48:05.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 Processor -- start 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 -- start start 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 0x7f2a3419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 0x7f2a3419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34111e60 con 0x7f2a34108010 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34111ce0 con 0x7f2a34107670 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.876+0000 7f2a3a200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34111fe0 con 0x7f2a34106540 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48354/0 (socket says 172.21.15.16:48354) 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 -- 172.21.15.16:0/594179746 learned_addr learned my addr 172.21.15.16:0/594179746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33400700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 0x7f2a3419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a32a00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 0x7f2a3419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 -- 172.21.15.16:0/594179746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 msgr2=0x7f2a3419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 0x7f2a3419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 -- 172.21.15.16:0/594179746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 msgr2=0x7f2a3419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 0x7f2a3419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33e00700 1 -- 172.21.15.16:0/594179746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a341a4480 con 0x7f2a34108010 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a33400700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 0x7f2a3419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:48:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.877+0000 7f2a32a00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 0x7f2a3419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:48:05.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a33e00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009010 tx=0x7f2a2c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:48:05.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c021020 con 0x7f2a34108010 2024-05-27T06:48:05.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a2c0054d0 con 0x7f2a34108010 2024-05-27T06:48:05.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c01b890 con 0x7f2a34108010 2024-05-27T06:48:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a341a4710 con 0x7f2a34108010 2024-05-27T06:48:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.878+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341a4b10 con 0x7f2a34108010 2024-05-27T06:48:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.880+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a340512b0 con 0x7f2a34108010 2024-05-27T06:48:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.882+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2a2c024020 con 0x7f2a34108010 2024-05-27T06:48:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.884+0000 7f2a2be00700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 0x7f2a180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.884+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f2a2c00f020 con 0x7f2a34108010 2024-05-27T06:48:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.884+0000 7f2a33400700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 0x7f2a180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.885+0000 7f2a33400700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 0x7f2a180b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a34107570 tx=0x7f2a2400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:48:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:05.887+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a2c0ab550 con 0x7f2a34108010 2024-05-27T06:48:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:06.143+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2a34063cb0 con 0x7f2a180b2900 2024-05-27T06:48:07.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:07 smithi069 ceph-mon[35276]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:07.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:07 smithi069 ceph-mon[35276]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:07.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:07.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:07.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:07.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:07 smithi016 ceph-mon[31415]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:07.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:07 smithi016 ceph-mon[31415]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:07.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:07.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:07.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:07 smithi164 ceph-mon[35327]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:07 smithi164 ceph-mon[35327]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:09.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:09 smithi016 ceph-mon[31415]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:09 smithi164 ceph-mon[35327]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:09 smithi069 ceph-mon[35276]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:11.580 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:11 smithi016 ceph-mon[31415]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:11.580 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:11 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]: dispatch 2024-05-27T06:48:11.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:11 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]': finished 2024-05-27T06:48:11.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:11 smithi016 ceph-mon[31415]: osdmap e22: 4 total, 3 up, 4 in 2024-05-27T06:48:11.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:11 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:11.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:11 smithi164 ceph-mon[35327]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:11.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:11 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]: dispatch 2024-05-27T06:48:11.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:11 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]': finished 2024-05-27T06:48:11.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:11 smithi164 ceph-mon[35327]: osdmap e22: 4 total, 3 up, 4 in 2024-05-27T06:48:11.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:11 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:11 smithi069 ceph-mon[35276]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:11.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:11 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]: dispatch 2024-05-27T06:48:11.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:11 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2413992376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1c5db9-adc3-444b-9eca-3599f6e32f89"}]': finished 2024-05-27T06:48:11.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:11 smithi069 ceph-mon[35276]: osdmap e22: 4 total, 3 up, 4 in 2024-05-27T06:48:11.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:11 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:12.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:12 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1244799323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:12.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:12 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1244799323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:12 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1244799323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:13.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:13 smithi016 ceph-mon[31415]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:13.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:13 smithi164 ceph-mon[35327]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:13 smithi069 ceph-mon[35276]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:15.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:15 smithi164 ceph-mon[35327]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:15.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:15 smithi016 ceph-mon[31415]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:15.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:15 smithi069 ceph-mon[35276]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:17 smithi164 ceph-mon[35327]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:17.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:17 smithi016 ceph-mon[31415]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:17.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:17 smithi069 ceph-mon[35276]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:19.530 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:19 smithi016 ceph-mon[31415]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:19 smithi164 ceph-mon[35327]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:19 smithi069 ceph-mon[35276]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:21.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:21 smithi164 ceph-mon[35327]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:21 smithi016 ceph-mon[31415]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:21 smithi069 ceph-mon[35276]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:22.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T06:48:22.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:22.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T06:48:22.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:22.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T06:48:22.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:23.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:23 smithi164 ceph-mon[35327]: Deploying daemon osd.3 on smithi016 2024-05-27T06:48:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:23 smithi164 ceph-mon[35327]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:23.440 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:23 smithi016 ceph-mon[31415]: Deploying daemon osd.3 on smithi016 2024-05-27T06:48:23.441 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:23 smithi016 ceph-mon[31415]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:23.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:23 smithi069 ceph-mon[35276]: Deploying daemon osd.3 on smithi016 2024-05-27T06:48:23.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:23 smithi069 ceph-mon[35276]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:25 smithi164 ceph-mon[35327]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:25 smithi016 ceph-mon[31415]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:25 smithi069 ceph-mon[35276]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:27 smithi164 ceph-mon[35327]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:27.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:27 smithi016 ceph-mon[31415]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:27 smithi069 ceph-mon[35276]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:29 smithi164 ceph-mon[35327]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:29 smithi016 ceph-mon[31415]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:29.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:29 smithi069 ceph-mon[35276]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:31 smithi016 ceph-mon[31415]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:31 smithi164 ceph-mon[35327]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:31.622 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:31 smithi069 ceph-mon[35276]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:32.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:32.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:32 smithi016 ceph-mon[31415]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:32.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:32 smithi069 ceph-mon[35276]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:32.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:32.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:32.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:32 smithi164 ceph-mon[35327]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:35 smithi016 ceph-mon[31415]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:35.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:35 smithi164 ceph-mon[35327]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:35.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:35 smithi069 ceph-mon[35276]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:36.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:36 smithi164 ceph-mon[35327]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T06:48:36.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:36 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:36.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:36 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:36 smithi016 ceph-mon[31415]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T06:48:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:36.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:36 smithi069 ceph-mon[35276]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T06:48:36.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:36 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:36.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:36 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:37.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T06:48:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T06:48:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:48:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:37.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.815+0000 7f2a2be00700 1 -- 172.21.15.16:0/594179746 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a34063cb0 con 0x7f2a180b2900 2024-05-27T06:48:37.809 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.817+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 msgr2=0x7f2a180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.817+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 0x7f2a180b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a34107570 tx=0x7f2a2400a680 comp rx=0 tx=0).stop 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 msgr2=0x7f2a341a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009010 tx=0x7f2a2c00bd70 comp rx=0 tx=0).stop 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 shutdown_connections 2024-05-27T06:48:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2a180b2900 0x7f2a180b4e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a34108010 0x7f2a341a3e90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106540 0x7f2a3419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 --2- 172.21.15.16:0/594179746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a34107670 0x7f2a3419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 >> 172.21.15.16:0/594179746 conn(0x7f2a34101b80 msgr2=0x7f2a3410a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.818+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 shutdown_connections 2024-05-27T06:48:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:48:37.819+0000 7f2a3a200700 1 -- 172.21.15.16:0/594179746 wait complete. 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:38.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' 2024-05-27T06:48:38.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] boot 2024-05-27T06:48:38.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: osdmap e25: 4 total, 4 up, 4 in 2024-05-27T06:48:38.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:48:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T06:48:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] boot 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: osdmap e25: 4 total, 4 up, 4 in 2024-05-27T06:48:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310]' entity='osd.3' 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: osd.3 [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] boot 2024-05-27T06:48:38.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: osdmap e25: 4 total, 4 up, 4 in 2024-05-27T06:48:38.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T06:48:38.919 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T06:48:38.924 INFO:tasks.cephadm:Deploying osd.4 on smithi069 with /dev/vg_nvme/lv_3... 2024-05-27T06:48:38.924 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T06:48:39.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:39 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:48:39.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:39 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:48:39.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:39 smithi016 ceph-mon[31415]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:39.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:39 smithi016 ceph-mon[31415]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T06:48:39.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:39 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:48:39.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:39 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:48:39.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:39 smithi164 ceph-mon[35327]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:39.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:39 smithi164 ceph-mon[35327]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T06:48:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:48:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:39 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:48:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:39 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:48:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:39 smithi069 ceph-mon[35276]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T06:48:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:39 smithi069 ceph-mon[35276]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T06:48:40.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:40 smithi069 ceph-mon[35276]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T06:48:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:40 smithi016 ceph-mon[31415]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T06:48:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:40 smithi164 ceph-mon[35327]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T06:48:41.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:41 smithi016 ceph-mon[31415]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-05-27T06:48:41.346 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:41 smithi069 ceph-mon[35276]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-05-27T06:48:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:41 smithi164 ceph-mon[35327]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0343293 s, 305 MB/s 2024-05-27T06:48:42.339 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-27T06:48:43.106 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-05-27T06:48:43.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:43 smithi016 ceph-mon[31415]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-27T06:48:43.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:43 smithi069 ceph-mon[35276]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-27T06:48:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:48:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:43 smithi164 ceph-mon[35327]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:45.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:45.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:48:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 -- 172.21.15.69:0/1677502093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810a780 msgr2=0x7ff76810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 --2- 172.21.15.69:0/1677502093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810a780 0x7ff76810ab90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff758009990 tx=0x7ff758009ca0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 -- 172.21.15.69:0/1677502093 shutdown_connections 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 --2- 172.21.15.69:0/1677502093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810c250 0x7ff768112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 --2- 172.21.15.69:0/1677502093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810a780 0x7ff76810ab90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 --2- 172.21.15.69:0/1677502093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 0x7ff76810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 -- 172.21.15.69:0/1677502093 >> 172.21.15.69:0/1677502093 conn(0x7ff768079830 msgr2=0x7ff7681085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:48:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 -- 172.21.15.69:0/1677502093 shutdown_connections 2024-05-27T06:48:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.715+0000 7ff767200700 1 -- 172.21.15.69:0/1677502093 wait complete. 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.716+0000 7ff767200700 1 Processor -- start 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 -- start start 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810a780 0x7ff7681a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 0x7ff7681a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681160c0 con 0x7ff76810a780 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff768115f40 con 0x7ff76810c250 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff767200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff768116240 con 0x7ff76810b8b0 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.717+0000 7ff766800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52990/0 (socket says 172.21.15.69:52990) 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 -- 172.21.15.69:0/3572681997 learned_addr learned my addr 172.21.15.69:0/3572681997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-27T06:48:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff765e00700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810a780 0x7ff7681a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:45.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff75fe00700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 0x7ff7681a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:45.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 -- 172.21.15.69:0/3572681997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 msgr2=0x7ff7681a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:45.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 0x7ff7681a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 -- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810a780 msgr2=0x7ff7681a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810a780 0x7ff7681a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 -- 172.21.15.69:0/3572681997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7681a8760 con 0x7ff76810c250 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.718+0000 7ff766800700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff760007ee0 tx=0x7ff76000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.719+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000dbd0 con 0x7ff76810c250 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.719+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff76000dd70 con 0x7ff76810c250 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.719+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681a8a50 con 0x7ff76810c250 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.719+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600058d0 con 0x7ff76810c250 2024-05-27T06:48:45.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.719+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681a8fb0 con 0x7ff76810c250 2024-05-27T06:48:45.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.720+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7680512b0 con 0x7ff76810c250 2024-05-27T06:48:45.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.723+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff76001a020 con 0x7ff76810c250 2024-05-27T06:48:45.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.724+0000 7ff75ea00700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 0x7ff74c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:48:45.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.724+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7ff7600e0e00 con 0x7ff76810c250 2024-05-27T06:48:45.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.725+0000 7ff765e00700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 0x7ff74c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:48:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.726+0000 7ff765e00700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 0x7ff74c0b4e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff758006010 tx=0x7ff75801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:48:45.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.729+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff760010070 con 0x7ff76810c250 2024-05-27T06:48:45.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:48:45.974+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff768063bf0 con 0x7ff74c0b2980 2024-05-27T06:48:46.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:46 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi016 2024-05-27T06:48:46.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:46 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 8547M 2024-05-27T06:48:46.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:46 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:46.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:46 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:46.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:46 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:46 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi016 2024-05-27T06:48:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:46 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 8547M 2024-05-27T06:48:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:46 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:46 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:46 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:46 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi016 2024-05-27T06:48:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:46 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 8547M 2024-05-27T06:48:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:46 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:48:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:46 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:48:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:46 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:48:47.220 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:47 smithi069 ceph-mon[35276]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T06:48:47.220 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:47 smithi069 ceph-mon[35276]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:47 smithi164 ceph-mon[35327]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T06:48:47.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:47 smithi164 ceph-mon[35327]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:47 smithi016 ceph-mon[31415]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T06:48:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:47 smithi016 ceph-mon[31415]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:48:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:49 smithi164 ceph-mon[35327]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:48:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:49 smithi016 ceph-mon[31415]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:48:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:49 smithi069 ceph-mon[35276]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:48:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:50 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/1944094962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:50 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:50 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]': finished 2024-05-27T06:48:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:50 smithi164 ceph-mon[35327]: osdmap e28: 5 total, 4 up, 5 in 2024-05-27T06:48:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:48:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:50 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/1944094962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:50 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:50 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]': finished 2024-05-27T06:48:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:50 smithi016 ceph-mon[31415]: osdmap e28: 5 total, 4 up, 5 in 2024-05-27T06:48:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:48:50.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:50 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/1944094962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:50 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]: dispatch 2024-05-27T06:48:50.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:50 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c25a93d3-53d0-4155-8981-50f926759f79"}]': finished 2024-05-27T06:48:50.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:50 smithi069 ceph-mon[35276]: osdmap e28: 5 total, 4 up, 5 in 2024-05-27T06:48:50.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:48:51.484 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:51 smithi164 ceph-mon[35327]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-27T06:48:51.484 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:51 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/538255109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:51 smithi016 ceph-mon[31415]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-27T06:48:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:51 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/538255109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:51.621 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:51 smithi069 ceph-mon[35276]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-27T06:48:51.621 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:51 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/538255109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:48:53.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:53 smithi164 ceph-mon[35327]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:53.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:53 smithi069 ceph-mon[35276]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:53 smithi016 ceph-mon[31415]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:55 smithi164 ceph-mon[35327]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:55 smithi016 ceph-mon[31415]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:55 smithi069 ceph-mon[35276]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:57.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:57 smithi069 ceph-mon[35276]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:57.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:57 smithi164 ceph-mon[35327]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:57.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:57 smithi016 ceph-mon[31415]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:48:59 smithi164 ceph-mon[35327]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:48:59 smithi016 ceph-mon[31415]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:48:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:48:59 smithi069 ceph-mon[35276]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:00.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T06:49:00.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T06:49:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T06:49:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:01.262 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:01 smithi069 ceph-mon[35276]: Deploying daemon osd.4 on smithi069 2024-05-27T06:49:01.262 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:01 smithi069 ceph-mon[35276]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:01 smithi164 ceph-mon[35327]: Deploying daemon osd.4 on smithi069 2024-05-27T06:49:01.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:01 smithi164 ceph-mon[35327]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:01 smithi016 ceph-mon[31415]: Deploying daemon osd.4 on smithi069 2024-05-27T06:49:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:01 smithi016 ceph-mon[31415]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:03 smithi069 ceph-mon[35276]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:03 smithi164 ceph-mon[35327]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:03.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:03 smithi016 ceph-mon[31415]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:05.487 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:05 smithi069 ceph-mon[35276]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:05 smithi164 ceph-mon[35327]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:05 smithi016 ceph-mon[31415]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:07 smithi164 ceph-mon[35327]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:07.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:07 smithi016 ceph-mon[31415]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:07 smithi069 ceph-mon[35276]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:09 smithi164 ceph-mon[35327]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:09 smithi016 ceph-mon[31415]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:09 smithi069 ceph-mon[35276]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:11.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:11 smithi164 ceph-mon[35327]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:11 smithi016 ceph-mon[31415]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:11 smithi069 ceph-mon[35276]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:13.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:13.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:14.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:13 smithi164 ceph-mon[35327]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:13 smithi164 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:13 smithi016 ceph-mon[31415]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:13 smithi016 ceph-mon[31415]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:13 smithi069 ceph-mon[35276]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:13 smithi069 ceph-mon[35276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T06:49:14.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.546+0000 7ff75ea00700 1 -- 172.21.15.69:0/3572681997 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff768063bf0 con 0x7ff74c0b2980 2024-05-27T06:49:14.540 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 4 on host 'smithi069' 2024-05-27T06:49:14.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 msgr2=0x7ff74c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 0x7ff74c0b4e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff758006010 tx=0x7ff75801a040 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 msgr2=0x7ff7681a8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff760007ee0 tx=0x7ff76000d3b0 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 shutdown_connections 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff74c0b2980 0x7ff74c0b4e90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76810c250 0x7ff7681a8120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76810a780 0x7ff7681a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 --2- 172.21.15.69:0/3572681997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76810b8b0 0x7ff7681a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:14.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.550+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 >> 172.21.15.69:0/3572681997 conn(0x7ff768079830 msgr2=0x7ff76810eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:49:14.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.551+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 shutdown_connections 2024-05-27T06:49:14.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:49:14.551+0000 7ff767200700 1 -- 172.21.15.69:0/3572681997 wait complete. 2024-05-27T06:49:15.314 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:15.314 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T06:49:15.314 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T06:49:15.314 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:15.314 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.315 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.315 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.315 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.315 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:15.366 DEBUG:teuthology.orchestra.run.smithi069:osd.4> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T06:49:15.372 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2024-05-27T06:49:15.372 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T06:49:15.801 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:49:15.968 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 06:49:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T06:49:15.813+0000 7f202ec00700 -1 osd.4 0 waiting for initial osdmap 2024-05-27T06:49:15.968 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 06:49:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T06:49:15.834+0000 7f2024000700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T06:49:16.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:16 smithi164 ceph-mon[35327]: from='osd.4 ' entity='osd.4' 2024-05-27T06:49:16.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:49:16.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:49:16.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:49:16.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T06:49:16.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:16 smithi016 ceph-mon[31415]: from='osd.4 ' entity='osd.4' 2024-05-27T06:49:16.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:49:16.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:49:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:49:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T06:49:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:16 smithi069 ceph-mon[35276]: from='osd.4 ' entity='osd.4' 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:17 smithi164 ceph-mon[35327]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:17 smithi164 ceph-mon[35327]: osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] boot 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:17 smithi164 ceph-mon[35327]: osdmap e31: 5 total, 5 up, 5 in 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:17.278 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:17 smithi069 ceph-mon[35276]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:17.279 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:17.279 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:17 smithi069 ceph-mon[35276]: osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] boot 2024-05-27T06:49:17.279 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:17 smithi069 ceph-mon[35276]: osdmap e31: 5 total, 5 up, 5 in 2024-05-27T06:49:17.279 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:17.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:17 smithi016 ceph-mon[31415]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T06:49:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:17 smithi016 ceph-mon[31415]: osd.4 [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] boot 2024-05-27T06:49:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:17 smithi016 ceph-mon[31415]: osdmap e31: 5 total, 5 up, 5 in 2024-05-27T06:49:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T06:49:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T06:49:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T06:49:18.559 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-27T06:49:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-27T06:49:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0361434 s, 290 MB/s 2024-05-27T06:49:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-27T06:49:19.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:18 smithi016 ceph-mon[31415]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T06:49:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:18 smithi016 ceph-mon[31415]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:18 smithi069 ceph-mon[35276]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T06:49:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:18 smithi069 ceph-mon[35276]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:19.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:18 smithi164 ceph-mon[35327]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T06:49:19.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:18 smithi164 ceph-mon[35327]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:19.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-05-27T06:49:19.761 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:21.221 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:21.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 -- 172.21.15.164:0/3144273196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 msgr2=0x7fe51010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3144273196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe51010bcc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009990 tx=0x7fe4fc009ca0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 -- 172.21.15.164:0/3144273196 shutdown_connections 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3144273196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 0x7fe510112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3144273196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 0x7fe51010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3144273196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe51010bcc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 -- 172.21.15.164:0/3144273196 >> 172.21.15.164:0/3144273196 conn(0x7fe510079830 msgr2=0x7fe5101085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 -- 172.21.15.164:0/3144273196 shutdown_connections 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.853+0000 7fe50fe00700 1 -- 172.21.15.164:0/3144273196 wait complete. 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.854+0000 7fe50fe00700 1 Processor -- start 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.854+0000 7fe50fe00700 1 -- start start 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 0x7fe5101a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:21.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:21.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 0x7fe5101a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:21.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101160a0 con 0x7fe51010a780 2024-05-27T06:49:21.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe510115f20 con 0x7fe51010c250 2024-05-27T06:49:21.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe510116220 con 0x7fe51010b8b0 2024-05-27T06:49:21.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:21.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46530/0 (socket says 172.21.15.164:46530) 2024-05-27T06:49:21.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.855+0000 7fe50e000700 1 -- 172.21.15.164:0/3863655792 learned_addr learned my addr 172.21.15.164:0/3863655792 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:49:21.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50ea00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 0x7fe5101a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50f400700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 0x7fe5101a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 -- 172.21.15.164:0/3863655792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 msgr2=0x7fe5101a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 0x7fe5101a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 -- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 msgr2=0x7fe5101a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 0x7fe5101a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 -- 172.21.15.164:0/3863655792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5101a87d0 con 0x7fe51010b8b0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.856+0000 7fe50e000700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc004790 tx=0x7fe4fc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.857+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc003ca0 con 0x7fe51010b8b0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.857+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4fc004830 con 0x7fe51010b8b0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.857+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005390 con 0x7fe51010b8b0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.857+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5101a8a60 con 0x7fe51010b8b0 2024-05-27T06:49:21.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.857+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a8e00 con 0x7fe51010b8b0 2024-05-27T06:49:21.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.858+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5100512b0 con 0x7fe51010b8b0 2024-05-27T06:49:21.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.861+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe4fc029060 con 0x7fe51010b8b0 2024-05-27T06:49:21.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.862+0000 7fe507400700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 0x7fe4f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:21.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.862+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fe4fc0e22a0 con 0x7fe51010b8b0 2024-05-27T06:49:21.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.863+0000 7fe50ea00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 0x7fe4f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:21.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.864+0000 7fe50ea00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 0x7fe4f40b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe500005950 tx=0x7fe5000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:49:21.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:21.866+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4fc013070 con 0x7fe51010b8b0 2024-05-27T06:49:22.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:22.109+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe510063bf0 con 0x7fe4f40b2880 2024-05-27T06:49:22.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:22 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi069 2024-05-27T06:49:22.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:22 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-27T06:49:22.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:22 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi069 2024-05-27T06:49:22.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:22 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-27T06:49:22.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:22 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi069 2024-05-27T06:49:22.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:22 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-27T06:49:23.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:23 smithi164 ceph-mon[35327]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:23 smithi164 ceph-mon[35327]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:49:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:49:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:49:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:23.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:23 smithi016 ceph-mon[31415]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:23.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:23 smithi016 ceph-mon[31415]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:49:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:49:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:49:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:23.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:23 smithi069 ceph-mon[35276]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:23 smithi069 ceph-mon[35276]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:49:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:49:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:49:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:25.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:25 smithi164 ceph-mon[35327]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:25.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:25 smithi016 ceph-mon[31415]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:25 smithi069 ceph-mon[35276]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/1625528683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]': finished 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: osdmap e33: 6 total, 5 up, 6 in 2024-05-27T06:49:27.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/1625528683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]': finished 2024-05-27T06:49:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: osdmap e33: 6 total, 5 up, 6 in 2024-05-27T06:49:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/1625528683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]: dispatch 2024-05-27T06:49:27.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e85ea8a0-f98e-416d-9d63-ed816d3afb04"}]': finished 2024-05-27T06:49:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: osdmap e33: 6 total, 5 up, 6 in 2024-05-27T06:49:27.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:28.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:28 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/3128894488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:49:28.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:28 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/3128894488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:49:28.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:28 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/3128894488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:49:29.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:29 smithi164 ceph-mon[35327]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:29 smithi016 ceph-mon[31415]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:29.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:29 smithi069 ceph-mon[35276]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:31.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:31 smithi069 ceph-mon[35276]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:31.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:31 smithi164 ceph-mon[35327]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:31 smithi016 ceph-mon[31415]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:33.146 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:33 smithi164 ceph-mon[35327]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:33.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:33 smithi016 ceph-mon[31415]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:33.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:33 smithi069 ceph-mon[35276]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:35 smithi016 ceph-mon[31415]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:35.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:35 smithi069 ceph-mon[35276]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:35.516 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:35 smithi164 ceph-mon[35327]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T06:49:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:36.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:36 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T06:49:36.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:36 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:36.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:36 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T06:49:36.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:36 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:37 smithi016 ceph-mon[31415]: Deploying daemon osd.5 on smithi164 2024-05-27T06:49:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:37 smithi016 ceph-mon[31415]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:37.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:37 smithi069 ceph-mon[35276]: Deploying daemon osd.5 on smithi164 2024-05-27T06:49:37.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:37 smithi069 ceph-mon[35276]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:37.370 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:37 smithi164 ceph-mon[35327]: Deploying daemon osd.5 on smithi164 2024-05-27T06:49:37.370 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:37 smithi164 ceph-mon[35327]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:39.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:39 smithi164 ceph-mon[35327]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:39 smithi016 ceph-mon[31415]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:39 smithi069 ceph-mon[35276]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:41 smithi069 ceph-mon[35276]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:41.419 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:41 smithi164 ceph-mon[35327]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:41 smithi016 ceph-mon[31415]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:43.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:43 smithi069 ceph-mon[35276]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:43.376 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:43 smithi164 ceph-mon[35327]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:43 smithi016 ceph-mon[31415]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:45 smithi164 ceph-mon[35327]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:45 smithi016 ceph-mon[31415]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:45 smithi069 ceph-mon[35276]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:47.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:47 smithi164 ceph-mon[35327]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:47 smithi016 ceph-mon[31415]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:47 smithi069 ceph-mon[35276]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:49.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:49.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.503 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:49.503 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:49.503 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.503 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:49.503 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:49 smithi164 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:49 smithi016 ceph-mon[31415]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:49.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:49 smithi069 ceph-mon[35276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T06:49:50.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.385+0000 7fe507400700 1 -- 172.21.15.164:0/3863655792 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe510063bf0 con 0x7fe4f40b2880 2024-05-27T06:49:50.379 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.388+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 msgr2=0x7fe4f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.388+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 0x7fe4f40b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe500005950 tx=0x7fe5000058e0 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.388+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 msgr2=0x7fe5101a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.388+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc004790 tx=0x7fe4fc009580 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 shutdown_connections 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe4f40b2880 0x7fe4f40b4d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51010c250 0x7fe5101a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51010a780 0x7fe5101a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 --2- 172.21.15.164:0/3863655792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe51010b8b0 0x7fe5101a11a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 >> 172.21.15.164:0/3863655792 conn(0x7fe510079830 msgr2=0x7fe51010eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:49:50.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 shutdown_connections 2024-05-27T06:49:50.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:49:50.389+0000 7fe50fe00700 1 -- 172.21.15.164:0/3863655792 wait complete. 2024-05-27T06:49:50.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T06:49:50.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T06:49:50.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:50.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.536 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:50.536 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.536 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.536 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:50.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:50.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:49:51.241 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T06:49:51.247 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-05-27T06:49:51.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T06:49:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:51 smithi016 ceph-mon[31415]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:49:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:51 smithi016 ceph-mon[31415]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T06:49:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:51 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:49:51.826 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:51 smithi164 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:49:51.826 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:51 smithi164 ceph-mon[35327]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T06:49:51.827 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:51 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:51.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:51 smithi069 ceph-mon[35276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:49:51.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:51 smithi069 ceph-mon[35276]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T06:49:51.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:51 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:52.763 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:49:52.763 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:49:52.763 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] boot 2024-05-27T06:49:52.763 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: osdmap e36: 6 total, 6 up, 6 in 2024-05-27T06:49:52.764 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:52.764 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:52.764 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:52 smithi164 ceph-mon[35327]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T06:49:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:49:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:49:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] boot 2024-05-27T06:49:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: osdmap e36: 6 total, 6 up, 6 in 2024-05-27T06:49:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:52 smithi016 ceph-mon[31415]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T06:49:52.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:49:52.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:49:52.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: osd.5 [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] boot 2024-05-27T06:49:52.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: osdmap e36: 6 total, 6 up, 6 in 2024-05-27T06:49:52.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T06:49:52.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:52.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:52 smithi069 ceph-mon[35276]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T06:49:54.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:54 smithi164 ceph-mon[35327]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T06:49:54.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:54 smithi164 ceph-mon[35327]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:54.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:54 smithi016 ceph-mon[31415]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T06:49:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:54 smithi016 ceph-mon[31415]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:54.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:54 smithi069 ceph-mon[35276]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T06:49:54.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:54 smithi069 ceph-mon[35276]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0351053 s, 299 MB/s 2024-05-27T06:49:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-27T06:49:56.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-05-27T06:49:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:49:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi164 2024-05-27T06:49:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:56 smithi164 ceph-mon[35327]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-27T06:49:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi164 2024-05-27T06:49:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:57.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:56 smithi016 ceph-mon[31415]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-27T06:49:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi164 2024-05-27T06:49:57.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:49:57.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-27T06:49:57.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:49:57.373 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:49:57.373 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:49:57.373 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:56 smithi069 ceph-mon[35276]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-27T06:49:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 -- 172.21.15.16:0/3348168923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 msgr2=0x7f81c0106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 --2- 172.21.15.16:0/3348168923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c0106950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b0009ca0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 -- 172.21.15.16:0/3348168923 shutdown_connections 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 --2- 172.21.15.16:0/3348168923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0108010 0x7f81c010e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 --2- 172.21.15.16:0/3348168923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c0106950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 --2- 172.21.15.16:0/3348168923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0107670 0x7f81c0107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 -- 172.21.15.16:0/3348168923 >> 172.21.15.16:0/3348168923 conn(0x7f81c0101b80 msgr2=0x7f81c0104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 -- 172.21.15.16:0/3348168923 shutdown_connections 2024-05-27T06:49:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.887+0000 7f81c5e00700 1 -- 172.21.15.16:0/3348168923 wait complete. 2024-05-27T06:49:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 Processor -- start 2024-05-27T06:49:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 -- start start 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 0x7f81c019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0108010 0x7f81c01a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0111e40 con 0x7f81c0106540 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0111cc0 con 0x7f81c0108010 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.889+0000 7f81c5e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0111fc0 con 0x7f81c0107670 2024-05-27T06:49:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35972/0 (socket says 172.21.15.16:35972) 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 -- 172.21.15.16:0/1582968699 learned_addr learned my addr 172.21.15.16:0/1582968699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bf400700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0108010 0x7f81c01a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81be000700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 0x7f81c019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 -- 172.21.15.16:0/1582968699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 msgr2=0x7f81c019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 0x7f81c019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 -- 172.21.15.16:0/1582968699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0108010 msgr2=0x7f81c01a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:49:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0108010 0x7f81c01a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:49:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81bea00700 1 -- 172.21.15.16:0/1582968699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01a4550 con 0x7f81c0106540 2024-05-27T06:49:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.890+0000 7f81be000700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 0x7f81c019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:49:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81bea00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098e0 tx=0x7f81b000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:49:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0026070 con 0x7f81c0106540 2024-05-27T06:49:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81b0003d20 con 0x7f81c0106540 2024-05-27T06:49:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01a47e0 con 0x7f81c0106540 2024-05-27T06:49:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000fd80 con 0x7f81c0106540 2024-05-27T06:49:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.891+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01a4cc0 con 0x7f81c0106540 2024-05-27T06:49:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.893+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c00512b0 con 0x7f81c0106540 2024-05-27T06:49:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.896+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f81b0029070 con 0x7f81c0106540 2024-05-27T06:49:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.897+0000 7f81c4a00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:49:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.897+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f81b00e2560 con 0x7f81c0106540 2024-05-27T06:49:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.897+0000 7f81be000700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:49:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.898+0000 7f81be000700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 0x7f81a40b4d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81c019daa0 tx=0x7f81ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:49:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:58.900+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81b0013070 con 0x7f81c0106540 2024-05-27T06:49:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:49:59.153+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f81c0063bf0 con 0x7f81a40b2880 2024-05-27T06:49:59.147 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:49:59 smithi016 ceph-mon[31415]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-27T06:49:59.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:49:59 smithi164 ceph-mon[35327]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-27T06:49:59.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:49:59 smithi069 ceph-mon[35276]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-27T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:00 smithi164 ceph-mon[35327]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:00 smithi164 ceph-mon[35327]: overall HEALTH_OK 2024-05-27T06:50:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:00 smithi016 ceph-mon[31415]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:00 smithi016 ceph-mon[31415]: overall HEALTH_OK 2024-05-27T06:50:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:00 smithi069 ceph-mon[35276]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:00.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:00.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:00 smithi069 ceph-mon[35276]: overall HEALTH_OK 2024-05-27T06:50:01.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:01 smithi164 ceph-mon[35327]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:50:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:01 smithi016 ceph-mon[31415]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:50:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:01 smithi069 ceph-mon[35276]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:50:03.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:03 smithi164 ceph-mon[35327]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T06:50:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:03 smithi016 ceph-mon[31415]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T06:50:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:03 smithi069 ceph-mon[35276]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T06:50:04.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:04 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]: dispatch 2024-05-27T06:50:04.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:04 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]': finished 2024-05-27T06:50:04.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:04 smithi164 ceph-mon[35327]: osdmap e39: 7 total, 6 up, 7 in 2024-05-27T06:50:04.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:04.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:04 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]: dispatch 2024-05-27T06:50:04.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:04 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]': finished 2024-05-27T06:50:04.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:04 smithi016 ceph-mon[31415]: osdmap e39: 7 total, 6 up, 7 in 2024-05-27T06:50:04.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:04 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]: dispatch 2024-05-27T06:50:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:04 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4252049394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be746ea8-a314-4561-9bf5-bc7d7944ffce"}]': finished 2024-05-27T06:50:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:04 smithi069 ceph-mon[35276]: osdmap e39: 7 total, 6 up, 7 in 2024-05-27T06:50:04.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:05.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:05 smithi164 ceph-mon[35327]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:50:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:05 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2263566265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:05.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:05 smithi016 ceph-mon[31415]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:50:05.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:05 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2263566265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:05.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:05 smithi069 ceph-mon[35276]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:50:05.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:05 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2263566265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:07.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:07 smithi164 ceph-mon[35327]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:07 smithi016 ceph-mon[31415]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:07.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:07 smithi069 ceph-mon[35276]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:09.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:09 smithi164 ceph-mon[35327]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:09.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:09 smithi016 ceph-mon[31415]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:09.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:09 smithi069 ceph-mon[35276]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:11.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:11 smithi016 ceph-mon[31415]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:11.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:11 smithi164 ceph-mon[35327]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:11.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:11 smithi069 ceph-mon[35276]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:13.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:13 smithi164 ceph-mon[35327]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:13 smithi016 ceph-mon[31415]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:13.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:13 smithi069 ceph-mon[35276]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:15 smithi016 ceph-mon[31415]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T06:50:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:15.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:15 smithi069 ceph-mon[35276]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:15.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T06:50:15.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:15.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:15 smithi164 ceph-mon[35327]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:15.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T06:50:15.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:16.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:16 smithi016 ceph-mon[31415]: Deploying daemon osd.6 on smithi016 2024-05-27T06:50:16.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:16 smithi069 ceph-mon[35276]: Deploying daemon osd.6 on smithi016 2024-05-27T06:50:16.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:16 smithi164 ceph-mon[35327]: Deploying daemon osd.6 on smithi016 2024-05-27T06:50:17.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:17 smithi016 ceph-mon[31415]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:17.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:17 smithi069 ceph-mon[35276]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:17 smithi164 ceph-mon[35327]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:19.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:19 smithi016 ceph-mon[31415]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:19.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:19 smithi069 ceph-mon[35276]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:19 smithi164 ceph-mon[35327]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:21.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:21 smithi069 ceph-mon[35276]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:21.386 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:21 smithi164 ceph-mon[35327]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:21 smithi016 ceph-mon[31415]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:23 smithi164 ceph-mon[35327]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:23.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:23 smithi016 ceph-mon[31415]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:23 smithi069 ceph-mon[35276]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:25.454 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:25 smithi016 ceph-mon[31415]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:25.454 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:25.454 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:25.454 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:25 smithi164 ceph-mon[35327]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:25 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:25 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:25 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:25 smithi069 ceph-mon[35276]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:25 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:25 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:25 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:27.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:27 smithi016 ceph-mon[31415]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:27 smithi164 ceph-mon[35327]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:27 smithi069 ceph-mon[35276]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T06:50:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:29.423 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:29.423 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:29 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:29.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:29 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:29.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:29 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:30.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T06:50:30.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T06:50:30.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:50:30.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:30.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:50:30.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:30 smithi016 ceph-mon[31415]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:50:30.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:30 smithi164 ceph-mon[35327]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T06:50:30.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T06:50:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T06:50:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:50:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:50:30.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:30 smithi069 ceph-mon[35276]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T06:50:30.651 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-05-27T06:50:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.658+0000 7f81c4a00700 1 -- 172.21.15.16:0/1582968699 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81c0063bf0 con 0x7f81a40b2880 2024-05-27T06:50:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.662+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 msgr2=0x7f81a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:50:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.662+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 0x7f81a40b4d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81c019daa0 tx=0x7f81ac009260 comp rx=0 tx=0).stop 2024-05-27T06:50:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.662+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 msgr2=0x7f81c019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.662+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098e0 tx=0x7f81b000ba90 comp rx=0 tx=0).stop 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 shutdown_connections 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0108010 0x7f81c01a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c0106540 0x7f81c019ca40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 --2- 172.21.15.16:0/1582968699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81c0107670 0x7f81c019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.663+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 >> 172.21.15.16:0/1582968699 conn(0x7f81c0101b80 msgr2=0x7f81c010a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.664+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 shutdown_connections 2024-05-27T06:50:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:50:30.664+0000 7f81c5e00700 1 -- 172.21.15.16:0/1582968699 wait complete. 2024-05-27T06:50:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:31 smithi016 ceph-mon[31415]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:31 smithi164 ceph-mon[35327]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' 2024-05-27T06:50:31.612 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.6.service 2024-05-27T06:50:31.617 INFO:tasks.cephadm:Deploying osd.7 on smithi069 with /dev/vg_nvme/lv_2... 2024-05-27T06:50:31.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:31 smithi069 ceph-mon[35276]: from='osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901]' entity='osd.6' 2024-05-27T06:50:32.036 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] boot 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: osdmap e42: 7 total, 7 up, 7 in 2024-05-27T06:50:32.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:32.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:50:32.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:50:32.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:32.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] boot 2024-05-27T06:50:32.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: osdmap e42: 7 total, 7 up, 7 in 2024-05-27T06:50:32.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:32.456 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:50:32.457 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:50:32.457 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:32.457 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: osd.6 [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] boot 2024-05-27T06:50:32.457 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: osdmap e42: 7 total, 7 up, 7 in 2024-05-27T06:50:32.458 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T06:50:33.253 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:33 smithi069 ceph-mon[35276]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:33.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:33 smithi164 ceph-mon[35327]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:33.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:33 smithi016 ceph-mon[31415]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T06:50:34.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:34 smithi069 ceph-mon[35276]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T06:50:34.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:34 smithi164 ceph-mon[35327]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T06:50:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:34 smithi016 ceph-mon[31415]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T06:50:34.981 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T06:50:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T06:50:34.982 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-27T06:50:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-27T06:50:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0363378 s, 289 MB/s 2024-05-27T06:50:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-27T06:50:35.327 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:35 smithi069 ceph-mon[35276]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:35 smithi164 ceph-mon[35327]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:35 smithi016 ceph-mon[31415]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:35.784 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-05-27T06:50:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:50:37.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:38.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.519+0000 7f8d9f400700 1 -- 172.21.15.69:0/1929278260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 msgr2=0x7f8d9810c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:50:38.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.519+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1929278260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d9810c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009990 tx=0x7f8d90009ca0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 -- 172.21.15.69:0/1929278260 shutdown_connections 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1929278260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d9810c6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1929278260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d98104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1929278260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d98105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 -- 172.21.15.69:0/1929278260 >> 172.21.15.69:0/1929278260 conn(0x7f8d980ff9b0 msgr2=0x7f8d98101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 -- 172.21.15.69:0/1929278260 shutdown_connections 2024-05-27T06:50:38.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.520+0000 7f8d9f400700 1 -- 172.21.15.69:0/1929278260 wait complete. 2024-05-27T06:50:38.514 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:38 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi016 2024-05-27T06:50:38.514 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:38 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 5698M 2024-05-27T06:50:38.515 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:38.515 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:38.515 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 Processor -- start 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 -- start start 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d9819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d9819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9810fc40 con 0x7f8d98105460 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9810fac0 con 0x7f8d98105e00 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9f400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9810fdc0 con 0x7f8d98104330 2024-05-27T06:50:38.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.522+0000 7f8d9d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52578/0 (socket says 172.21.15.69:52578) 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 -- 172.21.15.69:0/1540907905 learned_addr learned my addr 172.21.15.69:0/1540907905 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d97e00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d9819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9ca00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d9819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 -- 172.21.15.69:0/1540907905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 msgr2=0x7f8d9819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d9819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 -- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 msgr2=0x7f8d9819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d9819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 -- 172.21.15.69:0/1540907905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d981a2310 con 0x7f8d98105e00 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d97e00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d9819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:50:38.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9ca00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d9819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.523+0000 7f8d9d400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d90003820 tx=0x7f8d90009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.524+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90027070 con 0x7f8d98105e00 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.524+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d900040a0 con 0x7f8d98105e00 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.524+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d981a25a0 con 0x7f8d98105e00 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.524+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90022da0 con 0x7f8d98105e00 2024-05-27T06:50:38.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.524+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d981a29f0 con 0x7f8d98105e00 2024-05-27T06:50:38.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.525+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d980512b0 con 0x7f8d98105e00 2024-05-27T06:50:38.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.528+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8d9002a070 con 0x7f8d98105e00 2024-05-27T06:50:38.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.530+0000 7f8d95600700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 0x7f8d800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:50:38.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.530+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f8d900e2b90 con 0x7f8d98105e00 2024-05-27T06:50:38.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.530+0000 7f8d9ca00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 0x7f8d800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:50:38.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.532+0000 7f8d9ca00700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 0x7f8d800b4d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d98105360 tx=0x7f8d8c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:50:38.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.533+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d90031080 con 0x7f8d98105e00 2024-05-27T06:50:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:38 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi016 2024-05-27T06:50:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:38 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 5698M 2024-05-27T06:50:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:38 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi016 2024-05-27T06:50:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:38 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 5698M 2024-05-27T06:50:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:50:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:50:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:50:38.774+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8d98063bf0 con 0x7f8d800b2880 2024-05-27T06:50:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:39 smithi164 ceph-mon[35327]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:39 smithi016 ceph-mon[31415]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:39 smithi069 ceph-mon[35276]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:50:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:50:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:40.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:40 smithi069 ceph-mon[35276]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:40 smithi164 ceph-mon[35327]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:40 smithi016 ceph-mon[31415]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:50:41.535 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:41 smithi069 ceph-mon[35276]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:41 smithi164 ceph-mon[35327]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:41 smithi016 ceph-mon[31415]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/1241865605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]': finished 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: osdmap e44: 8 total, 7 up, 8 in 2024-05-27T06:50:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/1241865605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]': finished 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: osdmap e44: 8 total, 7 up, 8 in 2024-05-27T06:50:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:43 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/1241865605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]: dispatch 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4dced21-90cd-4a3d-8823-02824ca1c42a"}]': finished 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: osdmap e44: 8 total, 7 up, 8 in 2024-05-27T06:50:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:43 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:50:44.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/550957775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:44.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/550957775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/550957775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:50:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:45 smithi164 ceph-mon[35327]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:45 smithi016 ceph-mon[31415]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:45 smithi069 ceph-mon[35276]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:47.562 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:47 smithi069 ceph-mon[35276]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:47 smithi164 ceph-mon[35327]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:47 smithi016 ceph-mon[31415]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:49 smithi164 ceph-mon[35327]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:49.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:49 smithi069 ceph-mon[35276]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:49 smithi016 ceph-mon[31415]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:51 smithi164 ceph-mon[35327]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:51 smithi016 ceph-mon[31415]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:51 smithi069 ceph-mon[35276]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:53 smithi164 ceph-mon[35327]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:53.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T06:50:53.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:53 smithi016 ceph-mon[31415]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T06:50:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:53 smithi069 ceph-mon[35276]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T06:50:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:50:54.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:54 smithi164 ceph-mon[35327]: Deploying daemon osd.7 on smithi069 2024-05-27T06:50:54.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:54 smithi016 ceph-mon[31415]: Deploying daemon osd.7 on smithi069 2024-05-27T06:50:54.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:54 smithi069 ceph-mon[35276]: Deploying daemon osd.7 on smithi069 2024-05-27T06:50:55.498 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:55 smithi069 ceph-mon[35276]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:55 smithi164 ceph-mon[35327]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:55 smithi016 ceph-mon[31415]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:57.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:57 smithi164 ceph-mon[35327]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:57.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:57 smithi016 ceph-mon[31415]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:57.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:57 smithi069 ceph-mon[35276]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:50:59 smithi164 ceph-mon[35327]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:50:59 smithi016 ceph-mon[31415]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:50:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:50:59 smithi069 ceph-mon[35276]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:01 smithi016 ceph-mon[31415]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:01.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:01 smithi069 ceph-mon[35276]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:01 smithi164 ceph-mon[35327]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:03.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:03.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:02 smithi069 ceph-mon[35276]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:03.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:03.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:02 smithi164 ceph-mon[35327]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:02 smithi016 ceph-mon[31415]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:05.073 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:05 smithi069 ceph-mon[35276]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:05 smithi164 ceph-mon[35327]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:05 smithi016 ceph-mon[31415]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:06.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:06 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:06 smithi164 ceph-mon[35327]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:06.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:06 smithi164 ceph-mon[35327]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:06.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:06 smithi164 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:06.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:06 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:06 smithi016 ceph-mon[31415]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:06.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:06 smithi016 ceph-mon[31415]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:06.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:06 smithi016 ceph-mon[31415]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:06.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:06 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:06.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:06 smithi069 ceph-mon[35276]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:06.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:06 smithi069 ceph-mon[35276]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:06.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:06 smithi069 ceph-mon[35276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T06:51:07.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.504+0000 7f8d95600700 1 -- 172.21.15.69:0/1540907905 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d98063bf0 con 0x7f8d800b2880 2024-05-27T06:51:07.496 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 7 on host 'smithi069' 2024-05-27T06:51:07.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 msgr2=0x7f8d800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:07.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 0x7f8d800b4d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d98105360 tx=0x7f8d8c009040 comp rx=0 tx=0).stop 2024-05-27T06:51:07.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 msgr2=0x7f8d981a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d90003820 tx=0x7f8d90009580 comp rx=0 tx=0).stop 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 shutdown_connections 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8d800b2880 0x7f8d800b4d90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d98105e00 0x7f8d981a1c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d98104330 0x7f8d9819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 --2- 172.21.15.69:0/1540907905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d98105460 0x7f8d9819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.508+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 >> 172.21.15.69:0/1540907905 conn(0x7f8d980ff9b0 msgr2=0x7f8d98108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.509+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 shutdown_connections 2024-05-27T06:51:07.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:51:07.509+0000 7f8d9f400700 1 -- 172.21.15.69:0/1540907905 wait complete. 2024-05-27T06:51:07.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:07.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:08.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:08.342 DEBUG:teuthology.orchestra.run.smithi069:osd.7> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.7.service 2024-05-27T06:51:08.346 INFO:tasks.cephadm:Deploying osd.8 on smithi164 with /dev/vg_nvme/lv_2... 2024-05-27T06:51:08.347 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T06:51:08.597 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 06:51:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T06:51:08.459+0000 7f1456c00700 -1 osd.7 0 waiting for initial osdmap 2024-05-27T06:51:08.597 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 06:51:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T06:51:08.487+0000 7f144ca00700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T06:51:08.786 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:51:08.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:08 smithi164 ceph-mon[35327]: from='osd.7 ' entity='osd.7' 2024-05-27T06:51:08.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:08.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:08 smithi069 ceph-mon[35276]: from='osd.7 ' entity='osd.7' 2024-05-27T06:51:09.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:08 smithi016 ceph-mon[31415]: from='osd.7 ' entity='osd.7' 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] boot 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T06:51:09.739 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] boot 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T06:51:09.973 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:10.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:51:10.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:51:10.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:10.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: osd.7 [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] boot 2024-05-27T06:51:10.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T06:51:10.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T06:51:11.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:11 smithi164 ceph-mon[35327]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:11.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:11 smithi164 ceph-mon[35327]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T06:51:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:11 smithi016 ceph-mon[31415]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:11 smithi016 ceph-mon[31415]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T06:51:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:11 smithi069 ceph-mon[35276]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:11 smithi069 ceph-mon[35276]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T06:51:11.810 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T06:51:11.811 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T06:51:11.811 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-27T06:51:11.811 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-27T06:51:11.811 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0375305 s, 279 MB/s 2024-05-27T06:51:11.811 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-27T06:51:12.751 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-05-27T06:51:12.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:12 smithi164 ceph-mon[35327]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T06:51:12.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:12 smithi164 ceph-mon[35327]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:12.932 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:12 smithi069 ceph-mon[35276]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T06:51:12.932 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:12 smithi069 ceph-mon[35276]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:13.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:12 smithi016 ceph-mon[31415]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T06:51:13.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:12 smithi016 ceph-mon[31415]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:13.184 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi069 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:14.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:14.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:14 smithi164 ceph-mon[35327]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi069 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:14 smithi016 ceph-mon[31415]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi069 2024-05-27T06:51:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:14.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:14 smithi069 ceph-mon[35276]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:15.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 -- 172.21.15.164:0/3876349676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810ba70 msgr2=0x7fdbe810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:15.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3876349676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810ba70 0x7fdbe810be80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009990 tx=0x7fdbd4009ca0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 -- 172.21.15.164:0/3876349676 shutdown_connections 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3876349676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810c410 0x7fdbe8112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3876349676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3876349676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810ba70 0x7fdbe810be80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 -- 172.21.15.164:0/3876349676 >> 172.21.15.164:0/3876349676 conn(0x7fdbe80799b0 msgr2=0x7fdbe807be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 -- 172.21.15.164:0/3876349676 shutdown_connections 2024-05-27T06:51:15.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.528+0000 7fdbeec00700 1 -- 172.21.15.164:0/3876349676 wait complete. 2024-05-27T06:51:15.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.529+0000 7fdbeec00700 1 Processor -- start 2024-05-27T06:51:15.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.529+0000 7fdbeec00700 1 -- start start 2024-05-27T06:51:15.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:15.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810ba70 0x7fdbe81a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:15.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81162b0 con 0x7fdbe810ba70 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8116130 con 0x7fdbe810ab00 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbeec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8116430 con 0x7fdbe810c410 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbecc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbecc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48156/0 (socket says 172.21.15.164:48156) 2024-05-27T06:51:15.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.530+0000 7fdbecc00700 1 -- 172.21.15.164:0/3404524818 learned_addr learned my addr 172.21.15.164:0/3404524818 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:51:15.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbe7e00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:15.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbe7400700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810ba70 0x7fdbe81a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 -- 172.21.15.164:0/3404524818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 msgr2=0x7fdbe81a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe81a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 -- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810ba70 msgr2=0x7fdbe81a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810ba70 0x7fdbe81a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 -- 172.21.15.164:0/3404524818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe81a89b0 con 0x7fdbe810c410 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbe7e00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbecc00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0007ee0 tx=0x7fdbe000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.531+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ddd0 con 0x7fdbe810c410 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.532+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe81a8ca0 con 0x7fdbe810c410 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.532+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbe0004020 con 0x7fdbe810c410 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.532+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0015660 con 0x7fdbe810c410 2024-05-27T06:51:15.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.532+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81a9190 con 0x7fdbe810c410 2024-05-27T06:51:15.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.533+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe80512b0 con 0x7fdbe810c410 2024-05-27T06:51:15.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.535+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdbe00041d0 con 0x7fdbe810c410 2024-05-27T06:51:15.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.537+0000 7fdbe4c00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 0x7fdbcc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:15.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.537+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fdbe00e1a90 con 0x7fdbe810c410 2024-05-27T06:51:15.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.538+0000 7fdbe7e00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 0x7fdbcc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:15.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.539+0000 7fdbe7e00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 0x7fdbcc0b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005950 tx=0x7fdbd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:51:15.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.540+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbe0010070 con 0x7fdbe810c410 2024-05-27T06:51:15.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:15.781+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdbe8063cb0 con 0x7fdbcc0b2880 2024-05-27T06:51:16.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:16.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:16.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:16.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:16.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:16.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:16.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:16.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:17.037 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:16 smithi164 ceph-mon[35327]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:17.037 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:16 smithi164 ceph-mon[35327]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T06:51:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:16 smithi016 ceph-mon[31415]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:16 smithi016 ceph-mon[31415]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T06:51:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:16 smithi069 ceph-mon[35276]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:16 smithi069 ceph-mon[35276]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T06:51:19.588 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:19 smithi164 ceph-mon[35327]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:19 smithi016 ceph-mon[31415]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:19 smithi069 ceph-mon[35276]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:20.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:20 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/727199879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:20 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:20 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]': finished 2024-05-27T06:51:20.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:20 smithi164 ceph-mon[35327]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T06:51:20.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:20 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/727199879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:20 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:20 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]': finished 2024-05-27T06:51:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:20 smithi016 ceph-mon[31415]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T06:51:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:20 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/727199879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:20 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]: dispatch 2024-05-27T06:51:20.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:20 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe"}]': finished 2024-05-27T06:51:20.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:20 smithi069 ceph-mon[35276]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T06:51:20.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:21 smithi164 ceph-mon[35327]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T06:51:21.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:21 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/283505010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:51:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:21 smithi016 ceph-mon[31415]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T06:51:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:21 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/283505010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:51:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:21 smithi069 ceph-mon[35276]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T06:51:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:21 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/283505010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:51:23.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:23 smithi164 ceph-mon[35327]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:23 smithi016 ceph-mon[31415]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:23 smithi069 ceph-mon[35276]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:25.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:25 smithi164 ceph-mon[35327]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:25 smithi016 ceph-mon[31415]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:25 smithi069 ceph-mon[35276]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:27.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:27 smithi164 ceph-mon[35327]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:27.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:27 smithi016 ceph-mon[31415]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:27 smithi069 ceph-mon[35276]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:29.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:29 smithi164 ceph-mon[35327]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:29.422 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:29 smithi069 ceph-mon[35276]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:29 smithi016 ceph-mon[31415]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:30.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T06:51:30.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:30 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:30.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T06:51:30.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:30 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:30.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T06:51:30.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:30 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:31.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:31 smithi069 ceph-mon[35276]: Deploying daemon osd.8 on smithi164 2024-05-27T06:51:31.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:31 smithi069 ceph-mon[35276]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:31.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:31 smithi016 ceph-mon[31415]: Deploying daemon osd.8 on smithi164 2024-05-27T06:51:31.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:31 smithi016 ceph-mon[31415]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:31.561 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:31 smithi164 ceph-mon[35327]: Deploying daemon osd.8 on smithi164 2024-05-27T06:51:31.562 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:31 smithi164 ceph-mon[35327]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:33.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:33 smithi164 ceph-mon[35327]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:33 smithi016 ceph-mon[31415]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:33 smithi069 ceph-mon[35276]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:35 smithi164 ceph-mon[35327]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:35 smithi016 ceph-mon[31415]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:35 smithi069 ceph-mon[35276]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:37.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:37 smithi164 ceph-mon[35327]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:37 smithi016 ceph-mon[31415]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:37 smithi069 ceph-mon[35276]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:39 smithi164 ceph-mon[35327]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:39 smithi016 ceph-mon[31415]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:39 smithi069 ceph-mon[35276]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:41.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:41 smithi164 ceph-mon[35327]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:41 smithi016 ceph-mon[31415]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:41 smithi069 ceph-mon[35276]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:43.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:43 smithi069 ceph-mon[35276]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:43 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:43 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:43 smithi069 ceph-mon[35276]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:43.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:43 smithi069 ceph-mon[35276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:43.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:43 smithi164 ceph-mon[35327]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:43.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:43 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:43 smithi164 ceph-mon[35327]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:43.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:43 smithi164 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:43 smithi016 ceph-mon[31415]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:43 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:43 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:43 smithi016 ceph-mon[31415]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:43 smithi016 ceph-mon[31415]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T06:51:44.250 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 8 on host 'smithi164' 2024-05-27T06:51:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.257+0000 7fdbe4c00700 1 -- 172.21.15.164:0/3404524818 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbe8063cb0 con 0x7fdbcc0b2880 2024-05-27T06:51:44.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 msgr2=0x7fdbcc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:44.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 0x7fdbcc0b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005950 tx=0x7fdbd80058e0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 msgr2=0x7fdbe81a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:44.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0007ee0 tx=0x7fdbe000d3b0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 shutdown_connections 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdbcc0b2880 0x7fdbcc0b4d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe810c410 0x7fdbe81a8330 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe810ab00 0x7fdbe81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 --2- 172.21.15.164:0/3404524818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe810ba70 0x7fdbe81a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.262+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 >> 172.21.15.164:0/3404524818 conn(0x7fdbe80799b0 msgr2=0x7fdbe810eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.263+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 shutdown_connections 2024-05-27T06:51:44.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:51:44.263+0000 7fdbeec00700 1 -- 172.21.15.164:0/3404524818 wait complete. 2024-05-27T06:51:44.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:44.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:44 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T06:51:44.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:44 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T06:51:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:51:44.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:44 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:45.163 DEBUG:teuthology.orchestra.run.smithi164:osd.8> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.8.service 2024-05-27T06:51:45.167 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-05-27T06:51:45.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T06:51:45.347 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:45.347 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:51:45.347 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T06:51:45.347 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.347 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: from='osd.8 ' entity='osd.8' 2024-05-27T06:51:45.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] boot 2024-05-27T06:51:45.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T06:51:45.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: from='osd.8 ' entity='osd.8' 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] boot 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T06:51:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-27T06:51:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: from='osd.8 ' entity='osd.8' 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: osd.8 [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] boot 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T06:51:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T06:51:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:51:46.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:46 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:51:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:46 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:51:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:46 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:51:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:46 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:51:46.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:46 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:51:46.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:46 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:51:47.407 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:47 smithi164 ceph-mon[35327]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:47.407 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:47 smithi164 ceph-mon[35327]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T06:51:47.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:47 smithi016 ceph-mon[31415]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:47.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:47 smithi016 ceph-mon[31415]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T06:51:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:47 smithi069 ceph-mon[35276]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:47 smithi069 ceph-mon[35276]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T06:51:48.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:48 smithi164 ceph-mon[35327]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T06:51:48.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:48 smithi016 ceph-mon[31415]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T06:51:48.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:48 smithi069 ceph-mon[35276]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T06:51:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T06:51:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T06:51:48.966 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-27T06:51:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-27T06:51:48.966 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0590396 s, 178 MB/s 2024-05-27T06:51:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-27T06:51:49.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:49 smithi164 ceph-mon[35327]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:49.599 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:49 smithi016 ceph-mon[31415]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:49 smithi069 ceph-mon[35276]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 672 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:51:50.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-05-27T06:51:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi164 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-27T06:51:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:51.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:51.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:50 smithi016 ceph-mon[31415]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 675 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-27T06:51:51.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi164 2024-05-27T06:51:51.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:50 smithi164 ceph-mon[35327]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 675 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-27T06:51:51.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi164 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:51:51.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:51:51.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:50 smithi069 ceph-mon[35276]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 675 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-27T06:51:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 -- 172.21.15.16:0/1613470975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 msgr2=0x7fb84010e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 --2- 172.21.15.16:0/1613470975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb84010e500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb83800b0a0 tx=0x7fb83800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 -- 172.21.15.16:0/1613470975 shutdown_connections 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 --2- 172.21.15.16:0/1613470975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb84010e500 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 --2- 172.21.15.16:0/1613470975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 0x7fb840075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 --2- 172.21.15.16:0/1613470975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 0x7fb840076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 -- 172.21.15.16:0/1613470975 >> 172.21.15.16:0/1613470975 conn(0x7fb8400ff7a0 msgr2=0x7fb840101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 -- 172.21.15.16:0/1613470975 shutdown_connections 2024-05-27T06:51:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.883+0000 7fb844c00700 1 -- 172.21.15.16:0/1613470975 wait complete. 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.884+0000 7fb844c00700 1 Processor -- start 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 -- start start 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 0x7fb84019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 0x7fb84019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb840111c40 con 0x7fb84010bff0 2024-05-27T06:51:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb840111ac0 con 0x7fb8400760e0 2024-05-27T06:51:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb844c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb840111dc0 con 0x7fb8400776d0 2024-05-27T06:51:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb83fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb83fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57746/0 (socket says 172.21.15.16:57746) 2024-05-27T06:51:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.885+0000 7fb83fe00700 1 -- 172.21.15.16:0/727427640 learned_addr learned my addr 172.21.15.16:0/727427640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83f400700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 0x7fb84019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83ea00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 0x7fb84019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 -- 172.21.15.16:0/727427640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 msgr2=0x7fb84019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 0x7fb84019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 -- 172.21.15.16:0/727427640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 msgr2=0x7fb84019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 0x7fb84019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 -- 172.21.15.16:0/727427640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8401a4330 con 0x7fb84010bff0 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.886+0000 7fb83fe00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb838004d00 tx=0x7fb838004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.887+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838021070 con 0x7fb84010bff0 2024-05-27T06:51:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.887+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8401a45c0 con 0x7fb84010bff0 2024-05-27T06:51:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.887+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb838007b80 con 0x7fb84010bff0 2024-05-27T06:51:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.887+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380052a0 con 0x7fb84010bff0 2024-05-27T06:51:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.887+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401a4a70 con 0x7fb84010bff0 2024-05-27T06:51:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.888+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8400512b0 con 0x7fb84010bff0 2024-05-27T06:51:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.891+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb838024070 con 0x7fb84010bff0 2024-05-27T06:51:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.893+0000 7fb837e00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 0x7fb8240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:51:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.893+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7fb83800f020 con 0x7fb84010bff0 2024-05-27T06:51:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.893+0000 7fb83f400700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 0x7fb8240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:51:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.894+0000 7fb83f400700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 0x7fb8240b4e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8400773d0 tx=0x7fb830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:51:52.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:52.907+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb83802b020 con 0x7fb84010bff0 2024-05-27T06:51:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:51:53.157+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb840063bf0 con 0x7fb8240b2900 2024-05-27T06:51:53.150 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:53 smithi016 ceph-mon[31415]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-27T06:51:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:53 smithi164 ceph-mon[35327]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-27T06:51:53.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:53 smithi069 ceph-mon[35276]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-27T06:51:54.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:54 smithi164 ceph-mon[35327]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:54.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:54.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:54.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:54 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:54.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:54 smithi016 ceph-mon[31415]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:54.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:54.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:54.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:54 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:54.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:54 smithi069 ceph-mon[35276]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:51:54.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:51:54.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:51:54.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:54 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:51:55.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:55 smithi016 ceph-mon[31415]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:55.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:55 smithi069 ceph-mon[35276]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:55 smithi164 ceph-mon[35327]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T06:51:57.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:57 smithi016 ceph-mon[31415]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T06:51:57.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:57 smithi069 ceph-mon[35276]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T06:51:57.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:57 smithi164 ceph-mon[35327]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T06:51:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]: dispatch 2024-05-27T06:51:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]': finished 2024-05-27T06:51:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:58 smithi016 ceph-mon[31415]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T06:51:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:58 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:51:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]: dispatch 2024-05-27T06:51:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]': finished 2024-05-27T06:51:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:58 smithi069 ceph-mon[35276]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T06:51:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:58 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:51:58.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]: dispatch 2024-05-27T06:51:58.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/595703138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e4f0c06-7a28-44a5-8ee0-84370c455177"}]': finished 2024-05-27T06:51:58.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:58 smithi164 ceph-mon[35327]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T06:51:58.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:58 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:51:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:59 smithi016 ceph-mon[31415]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:51:59 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/420883677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:51:59.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:59 smithi069 ceph-mon[35276]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:59.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:51:59 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/420883677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:51:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:59 smithi164 ceph-mon[35327]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:51:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:51:59 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/420883677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:52:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:01 smithi016 ceph-mon[31415]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:01 smithi069 ceph-mon[35276]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:01 smithi164 ceph-mon[35327]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:03.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:03 smithi016 ceph-mon[31415]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:03 smithi069 ceph-mon[35276]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:03 smithi164 ceph-mon[35327]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:05.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:05 smithi069 ceph-mon[35276]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:05 smithi164 ceph-mon[35327]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:05 smithi016 ceph-mon[31415]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:07 smithi016 ceph-mon[31415]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:07 smithi164 ceph-mon[35327]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:07 smithi069 ceph-mon[35276]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:08.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T06:52:08.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:08.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T06:52:08.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T06:52:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:09.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:09 smithi016 ceph-mon[31415]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:09.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:09 smithi016 ceph-mon[31415]: Deploying daemon osd.9 on smithi016 2024-05-27T06:52:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:09 smithi164 ceph-mon[35327]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:09.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:09 smithi164 ceph-mon[35327]: Deploying daemon osd.9 on smithi016 2024-05-27T06:52:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:09 smithi069 ceph-mon[35276]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:09 smithi069 ceph-mon[35276]: Deploying daemon osd.9 on smithi016 2024-05-27T06:52:11.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:11 smithi016 ceph-mon[31415]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:11.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:11 smithi164 ceph-mon[35327]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:11 smithi069 ceph-mon[35276]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:13.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:13 smithi016 ceph-mon[31415]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:13.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:13 smithi164 ceph-mon[35327]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:13 smithi069 ceph-mon[35276]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:15.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:15 smithi164 ceph-mon[35327]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:15.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:15 smithi016 ceph-mon[31415]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:15.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:15 smithi069 ceph-mon[35276]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:17.533 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:17 smithi016 ceph-mon[31415]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:17 smithi164 ceph-mon[35327]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:17.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:17 smithi069 ceph-mon[35276]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:19.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:19 smithi016 ceph-mon[31415]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:19.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:19.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:19.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:19 smithi164 ceph-mon[35327]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:19.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:19.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:19 smithi069 ceph-mon[35276]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:21 smithi016 ceph-mon[31415]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:21 smithi164 ceph-mon[35327]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:21 smithi069 ceph-mon[35276]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:23.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:23 smithi164 ceph-mon[35327]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:23.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:23 smithi164 ceph-mon[35327]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T06:52:23.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:23.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:23 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:23 smithi016 ceph-mon[31415]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:23 smithi016 ceph-mon[31415]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T06:52:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:23 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:23 smithi069 ceph-mon[35276]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:23 smithi069 ceph-mon[35276]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T06:52:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:23 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.174+0000 7fb837e00700 1 -- 172.21.15.16:0/727427640 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb840063bf0 con 0x7fb8240b2900 2024-05-27T06:52:24.168 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.177+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 msgr2=0x7fb8240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.177+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 0x7fb8240b4e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8400773d0 tx=0x7fb830009260 comp rx=0 tx=0).stop 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.177+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 msgr2=0x7fb8401a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.177+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb838004d00 tx=0x7fb838004de0 comp rx=0 tx=0).stop 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 shutdown_connections 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb8240b2900 0x7fb8240b4e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84010bff0 0x7fb8401a3cb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8400760e0 0x7fb84019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 --2- 172.21.15.16:0/727427640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400776d0 0x7fb84019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 >> 172.21.15.16:0/727427640 conn(0x7fb8400ff7a0 msgr2=0x7fb840107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:52:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 shutdown_connections 2024-05-27T06:52:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:52:24.178+0000 7fb844c00700 1 -- 172.21.15.16:0/727427640 wait complete. 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:24.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:24.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:24.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:24.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:25.014 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.9.service 2024-05-27T06:52:25.019 INFO:tasks.cephadm:Deploying osd.10 on smithi069 with /dev/vg_nvme/lv_1... 2024-05-27T06:52:25.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T06:52:25.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:25.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:25 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:25.456 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:52:25.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:52:25.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:52:25.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:25.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:52:25.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T06:52:25.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:25.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:25 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: from='osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:25.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:25 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:26.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:26 smithi016 ceph-mon[31415]: osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] boot 2024-05-27T06:52:26.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:26 smithi016 ceph-mon[31415]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T06:52:26.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:26 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:26.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:26 smithi164 ceph-mon[35327]: osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] boot 2024-05-27T06:52:26.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:26 smithi164 ceph-mon[35327]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T06:52:26.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:26 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:26.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:26 smithi069 ceph-mon[35276]: osd.9 [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] boot 2024-05-27T06:52:26.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:26 smithi069 ceph-mon[35276]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T06:52:26.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:26 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T06:52:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:27 smithi016 ceph-mon[31415]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:27 smithi016 ceph-mon[31415]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T06:52:27.547 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:27 smithi069 ceph-mon[35276]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:27.548 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:27 smithi069 ceph-mon[35276]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T06:52:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:27 smithi164 ceph-mon[35327]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-27T06:52:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:27 smithi164 ceph-mon[35327]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T06:52:28.449 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T06:52:28.450 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T06:52:28.450 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-27T06:52:28.450 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-27T06:52:28.450 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0313361 s, 335 MB/s 2024-05-27T06:52:28.450 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-27T06:52:29.483 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-05-27T06:52:29.568 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:29 smithi069 ceph-mon[35276]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:29 smithi164 ceph-mon[35327]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:29 smithi016 ceph-mon[31415]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:29.923 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T06:52:31.302 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:31.302 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi016 2024-05-27T06:52:31.302 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.302 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.302 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:31.303 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi016 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-27T06:52:31.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:31.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:31.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi016 2024-05-27T06:52:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:52:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:32.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 -- 172.21.15.69:0/2610920691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 msgr2=0x7fc0a0104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 --2- 172.21.15.69:0/2610920691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a0104360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009990 tx=0x7fc08c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 -- 172.21.15.69:0/2610920691 shutdown_connections 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 --2- 172.21.15.69:0/2610920691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 0x7fc0a010c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 --2- 172.21.15.69:0/2610920691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 0x7fc0a0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 --2- 172.21.15.69:0/2610920691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a0104360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 -- 172.21.15.69:0/2610920691 >> 172.21.15.69:0/2610920691 conn(0x7fc0a00ff9c0 msgr2=0x7fc0a0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 -- 172.21.15.69:0/2610920691 shutdown_connections 2024-05-27T06:52:32.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.215+0000 7fc0a7600700 1 -- 172.21.15.69:0/2610920691 wait complete. 2024-05-27T06:52:32.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 Processor -- start 2024-05-27T06:52:32.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 -- start start 2024-05-27T06:52:32.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 0x7fc0a019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 0x7fc0a01a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010fd90 con 0x7fc0a01049d0 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010fc10 con 0x7fc0a0103f50 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.217+0000 7fc0a7600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010ff10 con 0x7fc0a01059a0 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60606/0 (socket says 172.21.15.69:60606) 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 -- 172.21.15.69:0/1416258030 learned_addr learned my addr 172.21.15.69:0/1416258030 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc097e00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 0x7fc0a019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:52:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a5600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 0x7fc0a01a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:52:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 -- 172.21.15.69:0/1416258030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 msgr2=0x7fc0a01a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:52:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 0x7fc0a01a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 -- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 msgr2=0x7fc0a019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:52:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 0x7fc0a019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:52:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 -- 172.21.15.69:0/1416258030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a01a4640 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.218+0000 7fc0a4c00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc08400d950 tx=0x7fc08400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.219+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084017070 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.219+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc08400de30 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.219+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a01a4930 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.219+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003aa0 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.219+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01a4e70 con 0x7fc0a0103f50 2024-05-27T06:52:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.220+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a00512b0 con 0x7fc0a0103f50 2024-05-27T06:52:32.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.222+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc084003c40 con 0x7fc0a0103f50 2024-05-27T06:52:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.224+0000 7fc09e000700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 0x7fc07c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:52:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.224+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7fc0840e1690 con 0x7fc0a0103f50 2024-05-27T06:52:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.225+0000 7fc097e00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 0x7fc07c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:52:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.226+0000 7fc097e00700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 0x7fc07c0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc08c006010 tx=0x7fc08c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:52:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.229+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc084020030 con 0x7fc0a0103f50 2024-05-27T06:52:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:52:32.470+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc0a00781f0 con 0x7fc07c0b2880 2024-05-27T06:52:33.452 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:33 smithi069 ceph-mon[35276]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:33.452 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:52:33.452 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:52:33.452 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:33 smithi164 ceph-mon[35327]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:52:33.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:52:33.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:33 smithi016 ceph-mon[31415]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:52:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:52:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:34.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:34 smithi069 ceph-mon[35276]: from='client.34343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:52:34.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:34 smithi164 ceph-mon[35327]: from='client.34343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:52:34.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:34 smithi016 ceph-mon[31415]: from='client.34343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:52:35.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:35 smithi069 ceph-mon[35276]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:35 smithi164 ceph-mon[35327]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:35 smithi016 ceph-mon[31415]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:37.487 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:37.487 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/1745992163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.487 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.487 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]': finished 2024-05-27T06:52:37.488 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T06:52:37.488 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:52:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/1745992163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]': finished 2024-05-27T06:52:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T06:52:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/1745992163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]: dispatch 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c705b502-5d5e-4b43-89db-40a14acec587"}]': finished 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T06:52:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:52:38.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:38 smithi164 ceph-mon[35327]: from='client.? 172.21.15.69:0/383970494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:52:38.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:38 smithi016 ceph-mon[31415]: from='client.? 172.21.15.69:0/383970494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:52:38.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:38 smithi069 ceph-mon[35276]: from='client.? 172.21.15.69:0/383970494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:52:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:39 smithi164 ceph-mon[35327]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:39 smithi016 ceph-mon[31415]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:39 smithi069 ceph-mon[35276]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:41.462 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:41 smithi069 ceph-mon[35276]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:41 smithi164 ceph-mon[35327]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:41 smithi016 ceph-mon[31415]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:43.480 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:43 smithi069 ceph-mon[35276]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:43 smithi164 ceph-mon[35327]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:43 smithi016 ceph-mon[31415]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:45.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:45 smithi069 ceph-mon[35276]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:45 smithi164 ceph-mon[35327]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:45 smithi016 ceph-mon[31415]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:47.489 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:47 smithi069 ceph-mon[35276]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:47.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:47 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T06:52:47.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:47 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:47 smithi164 ceph-mon[35327]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:47 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T06:52:47.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:47 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:47 smithi016 ceph-mon[31415]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:47 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T06:52:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:47 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:52:48.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:48 smithi164 ceph-mon[35327]: Deploying daemon osd.10 on smithi069 2024-05-27T06:52:48.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:48 smithi016 ceph-mon[31415]: Deploying daemon osd.10 on smithi069 2024-05-27T06:52:48.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:48 smithi069 ceph-mon[35276]: Deploying daemon osd.10 on smithi069 2024-05-27T06:52:49.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:49 smithi069 ceph-mon[35276]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:49 smithi164 ceph-mon[35327]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:49 smithi016 ceph-mon[31415]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:51.354 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:51 smithi069 ceph-mon[35276]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:51 smithi164 ceph-mon[35327]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:51 smithi016 ceph-mon[31415]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:53 smithi164 ceph-mon[35327]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:53 smithi016 ceph-mon[31415]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:53 smithi069 ceph-mon[35276]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:55.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:55 smithi069 ceph-mon[35276]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:55 smithi164 ceph-mon[35327]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:55 smithi016 ceph-mon[31415]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:57.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:57.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:56 smithi016 ceph-mon[31415]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:57.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:57.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:56 smithi069 ceph-mon[35276]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:57.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:52:57.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:52:57.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:56 smithi164 ceph-mon[35327]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:59.112 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:52:59 smithi069 ceph-mon[35276]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:52:59 smithi016 ceph-mon[31415]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:52:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:52:59 smithi164 ceph-mon[35327]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:00 smithi164 ceph-mon[35327]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:01.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:00 smithi016 ceph-mon[31415]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:01.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:01.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:01.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T06:53:01.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:00 smithi069 ceph-mon[35276]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.694+0000 7fc09e000700 1 -- 172.21.15.69:0/1416258030 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc0a00781f0 con 0x7fc07c0b2880 2024-05-27T06:53:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 10 on host 'smithi069' 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 msgr2=0x7fc07c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 0x7fc07c0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc08c006010 tx=0x7fc08c01a040 comp rx=0 tx=0).stop 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 msgr2=0x7fc0a019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc08400d950 tx=0x7fc08400dd10 comp rx=0 tx=0).stop 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 shutdown_connections 2024-05-27T06:53:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc07c0b2880 0x7fc07c0b4d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0a01059a0 0x7fc0a01a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a0103f50 0x7fc0a019cb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 --2- 172.21.15.69:0/1416258030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a01049d0 0x7fc0a019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 >> 172.21.15.69:0/1416258030 conn(0x7fc0a00ff9c0 msgr2=0x7fc0a0076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 shutdown_connections 2024-05-27T06:53:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-27T06:53:01.698+0000 7fc0a7600700 1 -- 172.21.15.69:0/1416258030 wait complete. 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:01.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:02.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:02.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:02.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:02.527 DEBUG:teuthology.orchestra.run.smithi069:osd.10> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.10.service 2024-05-27T06:53:02.533 INFO:tasks.cephadm:Deploying osd.11 on smithi164 with /dev/vg_nvme/lv_1... 2024-05-27T06:53:02.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T06:53:02.976 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:53:03.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:02 smithi164 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:53:03.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:02 smithi164 ceph-mon[35327]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T06:53:03.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:02 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:02 smithi164 ceph-mon[35327]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:03.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:02 smithi069 ceph-mon[35276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:53:03.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:02 smithi069 ceph-mon[35276]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T06:53:03.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:02 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:02 smithi069 ceph-mon[35276]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:03.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:02 smithi016 ceph-mon[31415]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-27T06:53:03.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:02 smithi016 ceph-mon[31415]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T06:53:03.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:02 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:03.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:02 smithi016 ceph-mon[31415]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-05-27T06:53:04.009 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:53:04.009 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:53:04.009 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] boot 2024-05-27T06:53:04.009 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T06:53:04.010 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:04.010 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:03 smithi069 ceph-mon[35276]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] boot 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:03 smithi164 ceph-mon[35327]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: osd.10 [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] boot 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T06:53:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:03 smithi016 ceph-mon[31415]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T06:53:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:05 smithi164 ceph-mon[35327]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:05 smithi164 ceph-mon[35327]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T06:53:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:05 smithi016 ceph-mon[31415]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:05 smithi016 ceph-mon[31415]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T06:53:05.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:05 smithi069 ceph-mon[35276]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:05.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:05 smithi069 ceph-mon[35276]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354665 s, 296 MB/s 2024-05-27T06:53:06.050 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-27T06:53:06.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:06 smithi164 ceph-mon[35327]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T06:53:06.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:06 smithi016 ceph-mon[31415]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T06:53:06.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:06 smithi069 ceph-mon[35276]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T06:53:06.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-05-27T06:53:07.325 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:53:07.516 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:07 smithi164 ceph-mon[35327]: pgmap v311: 1 pgs: 1 activating+degraded; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:07.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:07 smithi016 ceph-mon[31415]: pgmap v311: 1 pgs: 1 activating+degraded; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:07.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:07 smithi069 ceph-mon[35276]: pgmap v311: 1 pgs: 1 activating+degraded; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi069 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:08.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi069 2024-05-27T06:53:08.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:08.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi069 2024-05-27T06:53:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:08.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:09 smithi164 ceph-mon[35327]: pgmap v312: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:09.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 -- 172.21.15.164:0/1042296810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105c30 msgr2=0x7f37b810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:09.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 --2- 172.21.15.164:0/1042296810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105c30 0x7f37b810c4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b0a0 tx=0x7f37b000b3b0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 -- 172.21.15.164:0/1042296810 shutdown_connections 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 --2- 172.21.15.164:0/1042296810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105c30 0x7f37b810c4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 --2- 172.21.15.164:0/1042296810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 --2- 172.21.15.164:0/1042296810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105290 0x7f37b81056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 -- 172.21.15.164:0/1042296810 >> 172.21.15.164:0/1042296810 conn(0x7f37b80ff7a0 msgr2=0x7f37b8101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 -- 172.21.15.164:0/1042296810 shutdown_connections 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.603+0000 7f37be200700 1 -- 172.21.15.164:0/1042296810 wait complete. 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.604+0000 7f37be200700 1 Processor -- start 2024-05-27T06:53:09.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 -- start start 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 0x7f37b8076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810fb00 con 0x7f37b8104160 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810f980 con 0x7f37b8105c30 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37be200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810fc80 con 0x7f37b8105290 2024-05-27T06:53:09.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37b7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:09.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37b7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56430/0 (socket says 172.21.15.164:56430) 2024-05-27T06:53:09.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.605+0000 7f37b7e00700 1 -- 172.21.15.164:0/257775654 learned_addr learned my addr 172.21.15.164:0/257775654 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37bce00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37bd800700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 0x7f37b8076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 -- 172.21.15.164:0/257775654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 msgr2=0x7f37b8076040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 0x7f37b8076040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 -- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 msgr2=0x7f37b8077450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8077450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 -- 172.21.15.164:0/257775654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b80767a0 con 0x7f37b8105290 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37bce00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:53:09.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37bd800700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 0x7f37b8076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b7e00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f37a400d980 tx=0x7f37a400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4017070 con 0x7f37b8105290 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a4004830 con 0x7f37b8105290 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81a4740 con 0x7f37b8105290 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.606+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40053a0 con 0x7f37b8105290 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.607+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a4c20 con 0x7f37b8105290 2024-05-27T06:53:09.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.608+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b80512b0 con 0x7f37b8105290 2024-05-27T06:53:09.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.610+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f37a40049e0 con 0x7f37b8105290 2024-05-27T06:53:09.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.612+0000 7f37b5600700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 0x7f37a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:09.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.612+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f37a40e1f30 con 0x7f37b8105290 2024-05-27T06:53:09.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.613+0000 7f37bce00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 0x7f37a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:09.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.614+0000 7f37bce00700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 0x7f37a00b4e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005fd0 tx=0x7f37ac005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:09.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.615+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37a4020020 con 0x7f37b8105290 2024-05-27T06:53:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:09 smithi016 ceph-mon[31415]: pgmap v312: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:09 smithi069 ceph-mon[35276]: pgmap v312: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T06:53:09.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:09.857+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f37b802e9e0 con 0x7f37a00b2900 2024-05-27T06:53:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:53:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:53:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:10.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:53:10.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:53:10.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:10.785 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T06:53:10.785 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T06:53:10.786 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:11 smithi016 ceph-mon[31415]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:11 smithi016 ceph-mon[31415]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-27T06:53:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:11 smithi016 ceph-mon[31415]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T06:53:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:11 smithi016 ceph-mon[31415]: Cluster is now healthy 2024-05-27T06:53:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:11 smithi069 ceph-mon[35276]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:11 smithi069 ceph-mon[35276]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-27T06:53:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:11 smithi069 ceph-mon[35276]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T06:53:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:11 smithi069 ceph-mon[35276]: Cluster is now healthy 2024-05-27T06:53:11.648 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:11 smithi164 ceph-mon[35327]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:11.648 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:11 smithi164 ceph-mon[35327]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-27T06:53:11.648 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:11 smithi164 ceph-mon[35327]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T06:53:11.648 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:11 smithi164 ceph-mon[35327]: Cluster is now healthy 2024-05-27T06:53:13.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:13 smithi016 ceph-mon[31415]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T06:53:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:13 smithi069 ceph-mon[35276]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T06:53:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:13 smithi164 ceph-mon[35327]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T06:53:14.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:14 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/3747952410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:14 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:14 smithi069 ceph-mon[35276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]': finished 2024-05-27T06:53:14.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:14 smithi069 ceph-mon[35276]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T06:53:14.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:14.736 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:14 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/3747952410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.736 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:14 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.736 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:14 smithi164 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]': finished 2024-05-27T06:53:14.736 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:14 smithi164 ceph-mon[35327]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T06:53:14.736 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:14 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/3747952410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:14 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]: dispatch 2024-05-27T06:53:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:14 smithi016 ceph-mon[31415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b29f9e-85de-403e-bc32-f10ad4d089cd"}]': finished 2024-05-27T06:53:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:14 smithi016 ceph-mon[31415]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T06:53:14.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:15 smithi164 ceph-mon[35327]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:15 smithi164 ceph-mon[35327]: from='client.? 172.21.15.164:0/2492390875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:53:15.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:15 smithi016 ceph-mon[31415]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:15.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:15 smithi016 ceph-mon[31415]: from='client.? 172.21.15.164:0/2492390875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:53:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:15 smithi069 ceph-mon[35276]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:15 smithi069 ceph-mon[35276]: from='client.? 172.21.15.164:0/2492390875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T06:53:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:17 smithi164 ceph-mon[35327]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:17.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:17 smithi016 ceph-mon[31415]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:17 smithi069 ceph-mon[35276]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:18.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:18 smithi164 ceph-mon[35327]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:18.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:18 smithi016 ceph-mon[31415]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:18.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:18 smithi069 ceph-mon[35276]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T06:53:21.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:21 smithi164 ceph-mon[35327]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:21 smithi016 ceph-mon[31415]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:21.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:21 smithi069 ceph-mon[35276]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:23.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:23 smithi164 ceph-mon[35327]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:23.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:23 smithi016 ceph-mon[31415]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:23.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:23 smithi069 ceph-mon[35276]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:24.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T06:53:24.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:24 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:24.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T06:53:24.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:24 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:24.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T06:53:24.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:24 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:25.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:25 smithi164 ceph-mon[35327]: Deploying daemon osd.11 on smithi164 2024-05-27T06:53:25.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:25 smithi164 ceph-mon[35327]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:25 smithi016 ceph-mon[31415]: Deploying daemon osd.11 on smithi164 2024-05-27T06:53:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:25 smithi016 ceph-mon[31415]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:25 smithi069 ceph-mon[35276]: Deploying daemon osd.11 on smithi164 2024-05-27T06:53:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:25 smithi069 ceph-mon[35276]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:27.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:27 smithi164 ceph-mon[35327]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:27.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:27 smithi016 ceph-mon[31415]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:27 smithi069 ceph-mon[35276]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:29.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:29 smithi164 ceph-mon[35327]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:29 smithi016 ceph-mon[31415]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:29 smithi069 ceph-mon[35276]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:31.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:31 smithi069 ceph-mon[35276]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:31.492 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:31 smithi164 ceph-mon[35327]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:31 smithi016 ceph-mon[31415]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:33 smithi164 ceph-mon[35327]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:33 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:33 smithi016 ceph-mon[31415]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:33 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:33 smithi069 ceph-mon[35276]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:33 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:35.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:35 smithi164 ceph-mon[35327]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:35 smithi016 ceph-mon[31415]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:35 smithi069 ceph-mon[35276]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:37 smithi164 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:37 smithi016 ceph-mon[31415]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:37.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:37.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:37.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:37.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:37 smithi069 ceph-mon[35276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T06:53:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.940+0000 7f37b5600700 1 -- 172.21.15.164:0/257775654 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f37b802e9e0 con 0x7f37a00b2900 2024-05-27T06:53:38.934 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 11 on host 'smithi164' 2024-05-27T06:53:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 msgr2=0x7f37a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 0x7f37a00b4e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005fd0 tx=0x7f37ac005f00 comp rx=0 tx=0).stop 2024-05-27T06:53:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 msgr2=0x7f37b8075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f37a400d980 tx=0x7f37a400dd40 comp rx=0 tx=0).stop 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 shutdown_connections 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f37a00b2900 0x7f37a00b4e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8105c30 0x7f37b8076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b8104160 0x7f37b8077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 --2- 172.21.15.164:0/257775654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37b8105290 0x7f37b8075b00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.944+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 >> 172.21.15.164:0/257775654 conn(0x7f37b80ff7a0 msgr2=0x7f37b81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.945+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 shutdown_connections 2024-05-27T06:53:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:38.945+0000 7f37be200700 1 -- 172.21.15.164:0/257775654 wait complete. 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T06:53:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T06:53:39.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:39.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-27T06:53:39.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:39.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:39.751 DEBUG:teuthology.orchestra.run.smithi164:osd.11> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.11.service 2024-05-27T06:53:39.756 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-27T06:53:39.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T06:53:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:53:40.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:40 smithi164 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:53:40.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:40 smithi164 ceph-mon[35327]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T06:53:40.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:40 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:40.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:40 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:40 smithi016 ceph-mon[31415]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:53:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:40 smithi016 ceph-mon[31415]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T06:53:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:40 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:40 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:40.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:40 smithi069 ceph-mon[35276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-27T06:53:40.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:40 smithi069 ceph-mon[35276]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T06:53:40.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:40 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:40.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:40 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:41.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: purged_snaps scrub starts 2024-05-27T06:53:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: purged_snaps scrub ok 2024-05-27T06:53:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] boot 2024-05-27T06:53:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T06:53:41.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:41.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: purged_snaps scrub starts 2024-05-27T06:53:41.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: purged_snaps scrub ok 2024-05-27T06:53:41.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:41.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] boot 2024-05-27T06:53:41.341 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T06:53:41.341 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: purged_snaps scrub starts 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: purged_snaps scrub ok 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: osd.11 [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] boot 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T06:53:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T06:53:42.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:42 smithi016 ceph-mon[31415]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T06:53:42.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:42 smithi164 ceph-mon[35327]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T06:53:42.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:42 smithi069 ceph-mon[35276]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T06:53:42.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 -- 172.21.15.16:0/3231230484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88104360 msgr2=0x7eff88104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 --2- 172.21.15.16:0/3231230484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88104360 0x7eff88104770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff74009990 tx=0x7eff74009ca0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 -- 172.21.15.16:0/3231230484 shutdown_connections 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 --2- 172.21.15.16:0/3231230484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104d00 0x7eff8810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 --2- 172.21.15.16:0/3231230484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88103230 0x7eff88103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 --2- 172.21.15.16:0/3231230484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88104360 0x7eff88104770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 -- 172.21.15.16:0/3231230484 >> 172.21.15.16:0/3231230484 conn(0x7eff880fe870 msgr2=0x7eff88100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 -- 172.21.15.16:0/3231230484 shutdown_connections 2024-05-27T06:53:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.769+0000 7eff8fc00700 1 -- 172.21.15.16:0/3231230484 wait complete. 2024-05-27T06:53:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.770+0000 7eff8fc00700 1 Processor -- start 2024-05-27T06:53:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.770+0000 7eff8fc00700 1 -- start start 2024-05-27T06:53:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 0x7eff881056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88104d00 0x7eff88105be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810eb90 con 0x7eff88103230 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810ea10 con 0x7eff88104360 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8fc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810ed10 con 0x7eff88104d00 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49826/0 (socket says 172.21.15.16:49826) 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.771+0000 7eff8d200700 1 -- 172.21.15.16:0/442516996 learned_addr learned my addr 172.21.15.16:0/442516996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8dc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88104d00 0x7eff88105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff87e00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 0x7eff881056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 -- 172.21.15.16:0/442516996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88104d00 msgr2=0x7eff88105be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88104d00 0x7eff88105be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 -- 172.21.15.16:0/442516996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 msgr2=0x7eff881056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 0x7eff881056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 -- 172.21.15.16:0/442516996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff88106340 con 0x7eff88103230 2024-05-27T06:53:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff87e00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 0x7eff881056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.772+0000 7eff8d200700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff7c007f10 tx=0x7eff7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.773+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00dbd0 con 0x7eff88103230 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.773+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff7c00dd70 con 0x7eff88103230 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.773+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff881a49c0 con 0x7eff88103230 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.773+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c005a40 con 0x7eff88103230 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.773+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff881a4ef0 con 0x7eff88103230 2024-05-27T06:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.774+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff881a5120 con 0x7eff88103230 2024-05-27T06:53:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.777+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff7c01a020 con 0x7eff88103230 2024-05-27T06:53:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.779+0000 7eff85600700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 0x7eff700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.780+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7eff7c0e1ba0 con 0x7eff88103230 2024-05-27T06:53:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.780+0000 7eff87e00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 0x7eff700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.781+0000 7eff87e00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 0x7eff700b4e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff74006010 tx=0x7eff7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:42.783+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff7c010070 con 0x7eff88103230 2024-05-27T06:53:43.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.022+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7eff881a52f0 con 0x7eff88103230 2024-05-27T06:53:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.023+0000 7eff85600700 1 -- 172.21.15.16:0/442516996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7eff7c0a9cc0 con 0x7eff88103230 2024-05-27T06:53:43.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 msgr2=0x7eff700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 0x7eff700b4e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff74006010 tx=0x7eff7401a040 comp rx=0 tx=0).stop 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 msgr2=0x7eff88106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff7c007f10 tx=0x7eff7c00d3b0 comp rx=0 tx=0).stop 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 shutdown_connections 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7eff700b2900 0x7eff700b4e10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff88104d00 0x7eff88105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff88103230 0x7eff88106ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 --2- 172.21.15.16:0/442516996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff88104360 0x7eff881056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 >> 172.21.15.16:0/442516996 conn(0x7eff880fe870 msgr2=0x7eff88100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 shutdown_connections 2024-05-27T06:53:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:43.026+0000 7eff8fc00700 1 -- 172.21.15.16:0/442516996 wait complete. 2024-05-27T06:53:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:43 smithi016 ceph-mon[31415]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/442516996' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T06:53:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:43 smithi164 ceph-mon[35327]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:43.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/442516996' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T06:53:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:43 smithi069 ceph-mon[35276]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/442516996' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T06:53:44.001 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1716792820,"num_in_osds":12,"osd_in_since":1716792794,"num_remapped_pgs":0} 2024-05-27T06:53:44.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:53:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: Detected new or changed devices on smithi164 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-27T06:53:45.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:45.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:45.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: Detected new or changed devices on smithi164 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: Detected new or changed devices on smithi164 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:45.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:46.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 -- 172.21.15.16:0/4077810476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c10c5b0 msgr2=0x7fb47c10c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 --2- 172.21.15.16:0/4077810476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c10c5b0 0x7fb47c10c9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009990 tx=0x7fb46c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 -- 172.21.15.16:0/4077810476 shutdown_connections 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 --2- 172.21.15.16:0/4077810476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 0x7fb47c113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 --2- 172.21.15.16:0/4077810476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c10c5b0 0x7fb47c10c9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 --2- 172.21.15.16:0/4077810476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c106570 0x7fb47c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 -- 172.21.15.16:0/4077810476 >> 172.21.15.16:0/4077810476 conn(0x7fb47c102070 msgr2=0x7fb47c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 -- 172.21.15.16:0/4077810476 shutdown_connections 2024-05-27T06:53:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.866+0000 7fb483000700 1 -- 172.21.15.16:0/4077810476 wait complete. 2024-05-27T06:53:46.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.868+0000 7fb483000700 1 Processor -- start 2024-05-27T06:53:46.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.868+0000 7fb483000700 1 -- start start 2024-05-27T06:53:46.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:46.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 0x7fb47c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:46.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10c5b0 0x7fb47c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:46.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c116570 con 0x7fb47c106570 2024-05-27T06:53:46.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1163f0 con 0x7fb47c106ec0 2024-05-27T06:53:46.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb483000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1166f0 con 0x7fb47c10c5b0 2024-05-27T06:53:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb47be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb47be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49854/0 (socket says 172.21.15.16:49854) 2024-05-27T06:53:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb47be00700 1 -- 172.21.15.16:0/2794833674 learned_addr learned my addr 172.21.15.16:0/2794833674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:53:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb47b400700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 0x7fb47c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.869+0000 7fb481000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10c5b0 0x7fb47c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 -- 172.21.15.16:0/2794833674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10c5b0 msgr2=0x7fb47c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10c5b0 0x7fb47c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 -- 172.21.15.16:0/2794833674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 msgr2=0x7fb47c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 0x7fb47c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 -- 172.21.15.16:0/2794833674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a1df0 con 0x7fb47c106570 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb47be00700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c027070 con 0x7fb47c106570 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb46c00fc70 con 0x7fb47c106570 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a2080 con 0x7fb47c106570 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.870+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c022d30 con 0x7fb47c106570 2024-05-27T06:53:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.871+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a2560 con 0x7fb47c106570 2024-05-27T06:53:46.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.873+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c0512b0 con 0x7fb47c106570 2024-05-27T06:53:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.877+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb46c02a070 con 0x7fb47c106570 2024-05-27T06:53:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.878+0000 7fb478c00700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 0x7fb4600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.878+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb46c0e31c0 con 0x7fb47c106570 2024-05-27T06:53:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.879+0000 7fb47b400700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 0x7fb4600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.880+0000 7fb47b400700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 0x7fb4600b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb468005950 tx=0x7fb4680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:46.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:46.881+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb46c013070 con 0x7fb47c106570 2024-05-27T06:53:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.118+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb47c02e5e0 con 0x7fb47c106570 2024-05-27T06:53:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.120+0000 7fb478c00700 1 -- 172.21.15.16:0/2794833674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18422 (secure 0 0 0) 0x7fb46c0b1070 con 0x7fb47c106570 2024-05-27T06:53:47.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:53:47.113 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:53:41.111993+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:53:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 msgr2=0x7fb4600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 0x7fb4600b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb468005950 tx=0x7fb4680058e0 comp rx=0 tx=0).stop 2024-05-27T06:53:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 msgr2=0x7fb47c19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c009f90 comp rx=0 tx=0).stop 2024-05-27T06:53:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 shutdown_connections 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb4600b2880 0x7fb4600b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb47c10c5b0 0x7fb47c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb47c106570 0x7fb47c19cf20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 --2- 172.21.15.16:0/2794833674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c106ec0 0x7fb47c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.123+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 >> 172.21.15.16:0/2794833674 conn(0x7fb47c102070 msgr2=0x7fb47c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.124+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 shutdown_connections 2024-05-27T06:53:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:47.124+0000 7fb483000700 1 -- 172.21.15.16:0/2794833674 wait complete. 2024-05-27T06:53:47.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:47 smithi016 ceph-mon[31415]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:47.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:47 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2794833674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:53:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:47 smithi164 ceph-mon[35327]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:47 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2794833674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:53:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:47 smithi069 ceph-mon[35276]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:47 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2794833674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:53:47.977 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T06:47:58.074809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T06:53:47.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:53:49.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:49 smithi016 ceph-mon[31415]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:49 smithi164 ceph-mon[35327]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:49 smithi069 ceph-mon[35276]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 -- 172.21.15.16:0/2975923291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588115360 msgr2=0x7fb588117780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2975923291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588115360 0x7fb588117780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb580009990 tx=0x7fb580009ca0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 -- 172.21.15.16:0/2975923291 shutdown_connections 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2975923291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588115360 0x7fb588117780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2975923291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588075b50 0x7fb588075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2975923291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 0x7fb588114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.947+0000 7fb58fe00700 1 -- 172.21.15.16:0/2975923291 >> 172.21.15.16:0/2975923291 conn(0x7fb588100030 msgr2=0x7fb588102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.948+0000 7fb58fe00700 1 -- 172.21.15.16:0/2975923291 shutdown_connections 2024-05-27T06:53:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.948+0000 7fb58fe00700 1 -- 172.21.15.16:0/2975923291 wait complete. 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.949+0000 7fb58fe00700 1 Processor -- start 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.949+0000 7fb58fe00700 1 -- start start 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.949+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 0x7fb5881a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588115360 0x7fb5881a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58fe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811a940 con 0x7fb588075b50 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58fe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811a7c0 con 0x7fb588115360 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811aac0 con 0x7fb5880764a0 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49868/0 (socket says 172.21.15.16:49868) 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 -- 172.21.15.16:0/2316954575 learned_addr learned my addr 172.21.15.16:0/2316954575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58ca00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 0x7fb5881a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58de00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588115360 0x7fb5881a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 -- 172.21.15.16:0/2316954575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 msgr2=0x7fb5881a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 0x7fb5881a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 -- 172.21.15.16:0/2316954575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588115360 msgr2=0x7fb5881a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588115360 0x7fb5881a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.950+0000 7fb58d400700 1 -- 172.21.15.16:0/2316954575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5881a61a0 con 0x7fb588075b50 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb58d400700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0060b0 tx=0x7fb57c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c01a070 con 0x7fb588075b50 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb57c00de20 con 0x7fb588075b50 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c014690 con 0x7fb588075b50 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5881a6490 con 0x7fb588075b50 2024-05-27T06:53:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.951+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881addb0 con 0x7fb588075b50 2024-05-27T06:53:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.952+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880512b0 con 0x7fb588075b50 2024-05-27T06:53:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.955+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb57c01d070 con 0x7fb588075b50 2024-05-27T06:53:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.956+0000 7fb586000700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 0x7fb5700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.957+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb57c0e1bd0 con 0x7fb588075b50 2024-05-27T06:53:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.957+0000 7fb58ca00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 0x7fb5700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.958+0000 7fb58ca00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 0x7fb5700b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5881034e0 tx=0x7fb57400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:50.959+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb57c010070 con 0x7fb588075b50 2024-05-27T06:53:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.197+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb58802e5e0 con 0x7fb588075b50 2024-05-27T06:53:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.198+0000 7fb586000700 1 -- 172.21.15.16:0/2316954575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fb57c0a9cf0 con 0x7fb588075b50 2024-05-27T06:53:51.190 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-27T06:53:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 msgr2=0x7fb5700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 0x7fb5700b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5881034e0 tx=0x7fb57400b590 comp rx=0 tx=0).stop 2024-05-27T06:53:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 msgr2=0x7fb5881a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0060b0 tx=0x7fb57c007f90 comp rx=0 tx=0).stop 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 shutdown_connections 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb5700b2880 0x7fb5700b4d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb588115360 0x7fb5881a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588075b50 0x7fb5881a12d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 --2- 172.21.15.16:0/2316954575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5880764a0 0x7fb5881a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 >> 172.21.15.16:0/2316954575 conn(0x7fb588100030 msgr2=0x7fb588100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 shutdown_connections 2024-05-27T06:53:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:51.201+0000 7fb58fe00700 1 -- 172.21.15.16:0/2316954575 wait complete. 2024-05-27T06:53:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:51 smithi164 ceph-mon[35327]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:51 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2316954575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T06:53:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:51 smithi016 ceph-mon[31415]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:51 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2316954575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T06:53:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:51 smithi069 ceph-mon[35276]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:51 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2316954575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T06:53:52.044 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.b on smithi069 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.e on smithi069 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.h on smithi069 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.k on smithi069 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.c on smithi164 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.f on smithi164 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.i on smithi164 2024-05-27T06:53:52.045 INFO:tasks.cephadm:Adding mds.l on smithi164 2024-05-27T06:53:52.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l' 2024-05-27T06:53:52.516 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T06:53:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:53 smithi164 ceph-mon[35327]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:53 smithi016 ceph-mon[31415]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:53 smithi069 ceph-mon[35276]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 -- 172.21.15.164:0/3979439900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 msgr2=0x7f6b30107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 --2- 172.21.15.164:0/3979439900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b30107a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009990 tx=0x7f6b1c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 -- 172.21.15.164:0/3979439900 shutdown_connections 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 --2- 172.21.15.164:0/3979439900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30108010 0x7f6b3010e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 --2- 172.21.15.164:0/3979439900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30106540 0x7f6b30106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 --2- 172.21.15.164:0/3979439900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b30107a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.865+0000 7f6b35200700 1 -- 172.21.15.164:0/3979439900 >> 172.21.15.164:0/3979439900 conn(0x7f6b30101bc0 msgr2=0x7f6b30104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.866+0000 7f6b35200700 1 -- 172.21.15.164:0/3979439900 shutdown_connections 2024-05-27T06:53:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.866+0000 7f6b35200700 1 -- 172.21.15.164:0/3979439900 wait complete. 2024-05-27T06:53:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.867+0000 7f6b35200700 1 Processor -- start 2024-05-27T06:53:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.867+0000 7f6b35200700 1 -- start start 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 0x7f6b3019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30108010 0x7f6b301a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30111eb0 con 0x7f6b30106540 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30111d30 con 0x7f6b30108010 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b35200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30112030 con 0x7f6b30107670 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b2d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b2d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54888/0 (socket says 172.21.15.164:54888) 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b2d400700 1 -- 172.21.15.164:0/2384392607 learned_addr learned my addr 172.21.15.164:0/2384392607 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.868+0000 7f6b2de00700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 0x7f6b3019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2e800700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30108010 0x7f6b301a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 -- 172.21.15.164:0/2384392607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30108010 msgr2=0x7f6b301a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30108010 0x7f6b301a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 -- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 msgr2=0x7f6b3019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 0x7f6b3019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 -- 172.21.15.164:0/2384392607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b301a45f0 con 0x7f6b30107670 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2de00700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 0x7f6b3019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:53:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b2d400700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000c00 tx=0x7f6b1c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.869+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c003c10 con 0x7f6b30107670 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.870+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b1c004830 con 0x7f6b30107670 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.870+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c005350 con 0x7f6b30107670 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.870+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b301a4880 con 0x7f6b30107670 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.870+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b301a4cd0 con 0x7f6b30107670 2024-05-27T06:53:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.871+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b300512b0 con 0x7f6b30107670 2024-05-27T06:53:54.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.874+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6b1c028020 con 0x7f6b30107670 2024-05-27T06:53:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.876+0000 7f6b26a00700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 0x7f6b140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.876+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6b1c0e3220 con 0x7f6b30107670 2024-05-27T06:53:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.877+0000 7f6b2de00700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 0x7f6b140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.878+0000 7f6b2de00700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 0x7f6b140b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b20005950 tx=0x7f6b200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:54.879+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b1c02f070 con 0x7f6b30107670 2024-05-27T06:53:55.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.122+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}) -- 0x7f6b30063cf0 con 0x7f6b140b2900 2024-05-27T06:53:55.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.142+0000 7f6b26a00700 1 -- 172.21.15.164:0/2384392607 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6b30063cf0 con 0x7f6b140b2900 2024-05-27T06:53:55.136 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mds.all update... 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.145+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 msgr2=0x7f6b140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.145+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 0x7f6b140b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b20005950 tx=0x7f6b200058e0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.145+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 msgr2=0x7f6b3019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.145+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000c00 tx=0x7f6b1c0047b0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 shutdown_connections 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6b140b2900 0x7f6b140b4e10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b30108010 0x7f6b301a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b30106540 0x7f6b3019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 --2- 172.21.15.164:0/2384392607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b30107670 0x7f6b3019cfd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 >> 172.21.15.164:0/2384392607 conn(0x7f6b30101bc0 msgr2=0x7f6b3010a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:55.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 shutdown_connections 2024-05-27T06:53:55.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-27T06:53:55.146+0000 7f6b35200700 1 -- 172.21.15.164:0/2384392607 wait complete. 2024-05-27T06:53:55.537 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:55.537 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.537 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:55.537 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.537 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:55.538 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.538 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:55.538 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:55.538 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:55 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:55.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:55 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:53:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:55.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:55.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:55.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:55 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:55.925 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.a.service 2024-05-27T06:53:55.933 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service 2024-05-27T06:53:55.979 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.g.service 2024-05-27T06:53:56.024 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.j.service 2024-05-27T06:53:56.070 DEBUG:teuthology.orchestra.run.smithi069:mds.b> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.b.service 2024-05-27T06:53:56.077 DEBUG:teuthology.orchestra.run.smithi069:mds.e> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.e.service 2024-05-27T06:53:56.125 DEBUG:teuthology.orchestra.run.smithi069:mds.h> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.h.service 2024-05-27T06:53:56.170 DEBUG:teuthology.orchestra.run.smithi069:mds.k> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.k.service 2024-05-27T06:53:56.216 DEBUG:teuthology.orchestra.run.smithi164:mds.c> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service 2024-05-27T06:53:56.224 DEBUG:teuthology.orchestra.run.smithi164:mds.f> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.f.service 2024-05-27T06:53:56.269 DEBUG:teuthology.orchestra.run.smithi164:mds.i> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.i.service 2024-05-27T06:53:56.313 DEBUG:teuthology.orchestra.run.smithi164:mds.l> sudo journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.l.service 2024-05-27T06:53:56.360 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-27T06:53:56.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:53:56.485 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:56 smithi164 ceph-mon[35327]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:56.486 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:56 smithi164 ceph-mon[35327]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-05-27T06:53:56.486 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:56 smithi164 ceph-mon[35327]: Deploying daemon mds.l on smithi164 2024-05-27T06:53:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:56 smithi016 ceph-mon[31415]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:56 smithi016 ceph-mon[31415]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-05-27T06:53:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:56 smithi016 ceph-mon[31415]: Deploying daemon mds.l on smithi164 2024-05-27T06:53:56.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:56 smithi069 ceph-mon[35276]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:53:56.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:56 smithi069 ceph-mon[35276]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-05-27T06:53:56.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:56 smithi069 ceph-mon[35276]: Deploying daemon mds.l on smithi164 2024-05-27T06:53:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:53:57.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:57 smithi016 ceph-mon[31415]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:57.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:57 smithi164 ceph-mon[35327]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:57.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:57 smithi069 ceph-mon[35276]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:58.583 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:58 smithi164 systemd[1]: Starting Ceph mds.l for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:53:58.896 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:58 smithi164 podman[64406]: 2024-05-27 06:53:58.599248229 +0000 UTC m=+0.119407577 container create f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-27T06:53:58.896 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:58 smithi164 podman[64406]: 2024-05-27 06:53:58.515351969 +0000 UTC m=+0.035511280 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:53:58.896 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:58 smithi164 podman[64406]: 2024-05-27 06:53:58.903211906 +0000 UTC m=+0.423371278 container init f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T06:53:59.311 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:58 smithi164 podman[64406]: 2024-05-27 06:53:58.913946076 +0000 UTC m=+0.434105397 container start f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T06:53:59.311 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:59 smithi164 bash[64406]: f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 2024-05-27T06:53:59.311 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:59 smithi164 systemd[1]: Started Ceph mds.l for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:53:59.311 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mds[64425]: -- 172.21.15.164:0/1978219494 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a01f1da40 con 0x562a01f65600 2024-05-27T06:53:59.311 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 06:53:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l[64421]: starting mds.l at 2024-05-27T06:53:59.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 -- 172.21.15.16:0/1381882316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc108f10 msgr2=0x7f7ecc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:59.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/1381882316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc108f10 0x7f7ecc109320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80099c0 tx=0x7f7eb8009cd0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 -- 172.21.15.16:0/1381882316 shutdown_connections 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/1381882316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc109860 0x7f7ecc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/1381882316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/1381882316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc108f10 0x7f7ecc109320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 -- 172.21.15.16:0/1381882316 >> 172.21.15.16:0/1381882316 conn(0x7f7ecc100030 msgr2=0x7f7ecc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 -- 172.21.15.16:0/1381882316 shutdown_connections 2024-05-27T06:53:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.370+0000 7f7ed0c00700 1 -- 172.21.15.16:0/1381882316 wait complete. 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.372+0000 7f7ed0c00700 1 Processor -- start 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.372+0000 7f7ed0c00700 1 -- start start 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.372+0000 7f7ed0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.372+0000 7f7ed0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 0x7f7ecc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ed0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc109860 0x7f7ecc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ed0c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114390 con 0x7f7ecc109860 2024-05-27T06:53:59.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ed0c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114210 con 0x7f7ecc1045e0 2024-05-27T06:53:59.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ed0c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114510 con 0x7f7ecc108f10 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ec9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ec9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43146/0 (socket says 172.21.15.16:43146) 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7eca800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc109860 0x7f7ecc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ec9e00700 1 -- 172.21.15.16:0/178599323 learned_addr learned my addr 172.21.15.16:0/178599323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.373+0000 7f7ec9400700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 0x7f7ecc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 -- 172.21.15.16:0/178599323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 msgr2=0x7f7ecc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 0x7f7ecc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 -- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc109860 msgr2=0x7f7ecc1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc109860 0x7f7ecc1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 -- 172.21.15.16:0/178599323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1a1f30 con 0x7f7ecc1045e0 2024-05-27T06:53:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9400700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 0x7f7ecc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:53:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec9e00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc007f10 tx=0x7f7ebc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00dbd0 con 0x7f7ecc1045e0 2024-05-27T06:53:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.374+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1a2220 con 0x7f7ecc1045e0 2024-05-27T06:53:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.375+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ebc00dd70 con 0x7f7ecc1045e0 2024-05-27T06:53:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.375+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc005950 con 0x7f7ecc1045e0 2024-05-27T06:53:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.375+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a9b40 con 0x7f7ecc1045e0 2024-05-27T06:53:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.376+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc105a90 con 0x7f7ecc1045e0 2024-05-27T06:53:59.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.379+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7ebc01a020 con 0x7f7ecc1045e0 2024-05-27T06:53:59.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.381+0000 7f7ec2a00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 0x7f7eb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:53:59.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.381+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7ebc0e1bd0 con 0x7f7ecc1045e0 2024-05-27T06:53:59.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.381+0000 7f7ec9400700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 0x7f7eb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:53:59.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.382+0000 7f7ec9400700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 0x7f7eb00b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:53:59.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.394+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ebc010070 con 0x7f7ecc1045e0 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6832/1294360695,v1:172.21.15.164:6833/1294360695] up:boot 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: fsmap 1 up:standby 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:53:59 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6832/1294360695,v1:172.21.15.164:6833/1294360695] up:boot 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: fsmap 1 up:standby 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:53:59 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:53:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6832/1294360695,v1:172.21.15.164:6833/1294360695] up:boot 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: fsmap 1 up:standby 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:53:59.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:53:59 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:53:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.685+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ecc0512b0 con 0x7f7ecc1045e0 2024-05-27T06:53:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.686+0000 7f7ec2a00700 1 -- 172.21.15.16:0/178599323 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f7ebc0a9d90 con 0x7f7ecc1045e0 2024-05-27T06:53:59.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:53:59.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T06:53:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 msgr2=0x7f7eb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 0x7f7eb00b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb801a040 comp rx=0 tx=0).stop 2024-05-27T06:53:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 msgr2=0x7f7ecc19d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc007f10 tx=0x7f7ebc00d3b0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 shutdown_connections 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7eb00b2880 0x7f7eb00b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc109860 0x7f7ecc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ecc1045e0 0x7f7ecc19d060 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 --2- 172.21.15.16:0/178599323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ecc108f10 0x7f7ecc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 >> 172.21.15.16:0/178599323 conn(0x7f7ecc100030 msgr2=0x7f7ecc10a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.689+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 shutdown_connections 2024-05-27T06:53:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:53:59.690+0000 7f7ed0c00700 1 -- 172.21.15.16:0/178599323 wait complete. 2024-05-27T06:53:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-05-27T06:54:00.439 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:00 smithi069 ceph-mon[35276]: Deploying daemon mds.k on smithi069 2024-05-27T06:54:00.440 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:00 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/178599323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:00 smithi164 ceph-mon[35327]: Deploying daemon mds.k on smithi069 2024-05-27T06:54:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:00 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/178599323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:00.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:00 smithi016 ceph-mon[31415]: Deploying daemon mds.k on smithi069 2024-05-27T06:54:00.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:00 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/178599323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:01.569 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:01 smithi069 ceph-mon[35276]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:01 smithi164 ceph-mon[35327]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:01 smithi016 ceph-mon[31415]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:02.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:02.741 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:02 smithi069 systemd[1]: Starting Ceph mds.k for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:03.054 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:02 smithi069 podman[64544]: 2024-05-27 06:54:02.655523561 +0000 UTC m=+0.037779023 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:03.054 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:02 smithi069 podman[64544]: 2024-05-27 06:54:02.8159532 +0000 UTC m=+0.198208685 container create 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T06:54:03.054 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 podman[64544]: 2024-05-27 06:54:03.060659801 +0000 UTC m=+0.442915296 container init 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T06:54:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:03.340 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 podman[64544]: 2024-05-27 06:54:03.07019719 +0000 UTC m=+0.452452645 container start 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T06:54:03.341 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 bash[64544]: 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada 2024-05-27T06:54:03.341 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 systemd[1]: Started Ceph mds.k for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:03.341 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mds[64564]: -- 172.21.15.69:0/3256233796 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4948b5a40 con 0x55d4948fd180 2024-05-27T06:54:03.341 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 06:54:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k[64560]: starting mds.k at 2024-05-27T06:54:03.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:03.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:03.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:03.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:03.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:03.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:03.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:04.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:04 smithi016 ceph-mon[31415]: Deploying daemon mds.j on smithi016 2024-05-27T06:54:04.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:04 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:boot 2024-05-27T06:54:04.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:04 smithi016 ceph-mon[31415]: fsmap 2 up:standby 2024-05-27T06:54:04.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T06:54:04.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:04 smithi069 ceph-mon[35276]: Deploying daemon mds.j on smithi016 2024-05-27T06:54:04.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:04 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:boot 2024-05-27T06:54:04.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:04 smithi069 ceph-mon[35276]: fsmap 2 up:standby 2024-05-27T06:54:04.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T06:54:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:04 smithi164 ceph-mon[35327]: Deploying daemon mds.j on smithi016 2024-05-27T06:54:04.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:04 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:boot 2024-05-27T06:54:04.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:04 smithi164 ceph-mon[35327]: fsmap 2 up:standby 2024-05-27T06:54:04.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T06:54:05.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:05 smithi016 ceph-mon[31415]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:05.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:05 smithi069 ceph-mon[35276]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 -- 172.21.15.16:0/805948779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c108d20 msgr2=0x7f6d7c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 --2- 172.21.15.16:0/805948779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c108d20 0x7f6d7c109130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d680099c0 tx=0x7f6d68009cd0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 -- 172.21.15.16:0/805948779 shutdown_connections 2024-05-27T06:54:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 --2- 172.21.15.16:0/805948779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c109670 0x7f6d7c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 --2- 172.21.15.16:0/805948779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 --2- 172.21.15.16:0/805948779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c108d20 0x7f6d7c109130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 -- 172.21.15.16:0/805948779 >> 172.21.15.16:0/805948779 conn(0x7f6d7c079d00 msgr2=0x7f6d7c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 -- 172.21.15.16:0/805948779 shutdown_connections 2024-05-27T06:54:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.669+0000 7f6d7b200700 1 -- 172.21.15.16:0/805948779 wait complete. 2024-05-27T06:54:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.670+0000 7f6d7b200700 1 Processor -- start 2024-05-27T06:54:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 -- start start 2024-05-27T06:54:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c108d20 0x7f6d7c19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c114180 con 0x7f6d7c109670 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c114000 con 0x7f6d7c1043f0 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.671+0000 7f6d7b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c114300 con 0x7f6d7c108d20 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39722/0 (socket says 172.21.15.16:39722) 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 -- 172.21.15.16:0/3306590903 learned_addr learned my addr 172.21.15.16:0/3306590903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d79e00700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d79400700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c108d20 0x7f6d7c19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 -- 172.21.15.16:0/3306590903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c108d20 msgr2=0x7f6d7c19d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c108d20 0x7f6d7c19d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 -- 172.21.15.16:0/3306590903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 msgr2=0x7f6d7c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d7a800700 1 -- 172.21.15.16:0/3306590903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c1a1c80 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.672+0000 7f6d79e00700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d7a800700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ef30 tx=0x7f6d7400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740092e0 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d74004bb0 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74004080 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c1a1f10 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.673+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c1a23d0 con 0x7f6d7c109670 2024-05-27T06:54:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.674+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c1058a0 con 0x7f6d7c109670 2024-05-27T06:54:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.677+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6d7401b030 con 0x7f6d7c109670 2024-05-27T06:54:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.679+0000 7f6d72a00700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 0x7f6d600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.679+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6d740e22a0 con 0x7f6d7c109670 2024-05-27T06:54:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.679+0000 7f6d79e00700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 0x7f6d600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.680+0000 7f6d79e00700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 0x7f6d600b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005950 tx=0x7f6d6c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.681+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d74022020 con 0x7f6d7c109670 2024-05-27T06:54:05.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:05 smithi164 ceph-mon[35327]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.973+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d7c0512b0 con 0x7f6d7c109670 2024-05-27T06:54:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.974+0000 7f6d72a00700 1 -- 172.21.15.16:0/3306590903 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f6d7c0512b0 con 0x7f6d7c109670 2024-05-27T06:54:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T06:54:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 msgr2=0x7f6d600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 0x7f6d600b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005950 tx=0x7f6d6c0058e0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 msgr2=0x7f6d7c1a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ef30 tx=0x7f6d7400c450 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 shutdown_connections 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6d600b2900 0x7f6d600b4e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7c109670 0x7f6d7c1a15e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d7c1043f0 0x7f6d7c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 --2- 172.21.15.16:0/3306590903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c108d20 0x7f6d7c19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.977+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 >> 172.21.15.16:0/3306590903 conn(0x7f6d7c079d00 msgr2=0x7f6d7c10f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.978+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 shutdown_connections 2024-05-27T06:54:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:05.978+0000 7f6d7b200700 1 -- 172.21.15.16:0/3306590903 wait complete. 2024-05-27T06:54:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-05-27T06:54:06.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:06 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3306590903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:06.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:06 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3306590903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:06.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:06 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3306590903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:07.444 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:07 smithi016 ceph-mon[31415]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:07 smithi164 ceph-mon[35327]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:07.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:07 smithi069 ceph-mon[35276]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:08.111 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:07 smithi016 systemd[1]: Starting Ceph mds.j for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:08.480 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 podman[69917]: 2024-05-27 06:54:08.059205011 +0000 UTC m=+0.038593232 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:08.481 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 podman[69917]: 2024-05-27 06:54:08.165613711 +0000 UTC m=+0.145001959 container create 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T06:54:08.481 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 podman[69917]: 2024-05-27 06:54:08.487012084 +0000 UTC m=+0.466400355 container init 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T06:54:08.860 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 podman[69917]: 2024-05-27 06:54:08.498486124 +0000 UTC m=+0.477874346 container start 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T06:54:08.860 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 bash[69917]: 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 2024-05-27T06:54:08.860 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 ceph-mds[69936]: -- 172.21.15.16:0/2281465199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55793ca8ba40 con 0x55793cad3a80 2024-05-27T06:54:08.860 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j[69932]: starting mds.j at 2024-05-27T06:54:08.860 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 06:54:08 smithi016 systemd[1]: Started Ceph mds.j for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:09.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:09.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:09.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:09.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:09.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:09.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:10 smithi016 ceph-mon[31415]: Deploying daemon mds.d on smithi016 2024-05-27T06:54:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:10 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.16:6834/553407450,v1:172.21.15.16:6835/553407450] up:boot 2024-05-27T06:54:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:10 smithi016 ceph-mon[31415]: fsmap 3 up:standby 2024-05-27T06:54:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:10 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T06:54:10.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:10 smithi164 ceph-mon[35327]: Deploying daemon mds.d on smithi016 2024-05-27T06:54:10.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:10 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.16:6834/553407450,v1:172.21.15.16:6835/553407450] up:boot 2024-05-27T06:54:10.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:10 smithi164 ceph-mon[35327]: fsmap 3 up:standby 2024-05-27T06:54:10.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:10 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T06:54:10.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:10 smithi069 ceph-mon[35276]: Deploying daemon mds.d on smithi016 2024-05-27T06:54:10.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:10 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.16:6834/553407450,v1:172.21.15.16:6835/553407450] up:boot 2024-05-27T06:54:10.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:10 smithi069 ceph-mon[35276]: fsmap 3 up:standby 2024-05-27T06:54:10.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:10 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T06:54:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:11.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:11 smithi016 ceph-mon[31415]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:11 smithi164 ceph-mon[35327]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:11.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:11 smithi069 ceph-mon[35276]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:12.360 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:12 smithi016 systemd[1]: Starting Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:12 smithi164 ceph-mon[35327]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:12.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:12 smithi016 ceph-mon[31415]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:12.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:12 smithi069 ceph-mon[35276]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:13.111 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 podman[70314]: 2024-05-27 06:54:12.991182694 +0000 UTC m=+0.043123703 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:13.444 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 podman[70314]: 2024-05-27 06:54:13.141777624 +0000 UTC m=+0.193718711 container create 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 podman[70314]: 2024-05-27 06:54:13.451686248 +0000 UTC m=+0.503627342 container init 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 podman[70314]: 2024-05-27 06:54:13.462209576 +0000 UTC m=+0.514150630 container start 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 ceph-mds[70349]: -- 172.21.15.16:0/1524554934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9f04d9a40 con 0x55f9f0521a80 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: starting mds.d at 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 bash[70314]: 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb 2024-05-27T06:54:13.764 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 06:54:13 smithi016 systemd[1]: Started Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 -- 172.21.15.16:0/3892761403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106570 msgr2=0x7fbdf0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/3892761403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106570 0x7fbdf0106980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009990 tx=0x7fbddc009ca0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 -- 172.21.15.16:0/3892761403 shutdown_connections 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/3892761403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf0106ec0 0x7fbdf0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/3892761403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf010c5b0 0x7fbdf010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/3892761403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106570 0x7fbdf0106980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 -- 172.21.15.16:0/3892761403 >> 172.21.15.16:0/3892761403 conn(0x7fbdf0102070 msgr2=0x7fbdf01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 -- 172.21.15.16:0/3892761403 shutdown_connections 2024-05-27T06:54:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.482+0000 7fbdf5a00700 1 -- 172.21.15.16:0/3892761403 wait complete. 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.483+0000 7fbdf5a00700 1 Processor -- start 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.483+0000 7fbdf5a00700 1 -- start start 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf010c5b0 0x7fbdf01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0116590 con 0x7fbdf0106ec0 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0116410 con 0x7fbdf0106570 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdf5a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0116710 con 0x7fbdf010c5b0 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdee000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdeea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdeea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41526/0 (socket says 172.21.15.16:41526) 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdee000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50166/0 (socket says 172.21.15.16:50166) 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.484+0000 7fbdeea00700 1 -- 172.21.15.16:0/4027366398 learned_addr learned my addr 172.21.15.16:0/4027366398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdef400700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf010c5b0 0x7fbdf01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 -- 172.21.15.16:0/4027366398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf010c5b0 msgr2=0x7fbdf01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf010c5b0 0x7fbdf01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 -- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 msgr2=0x7fbdf019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 -- 172.21.15.16:0/4027366398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf01a1f10 con 0x7fbdf0106570 2024-05-27T06:54:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdee000700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.485+0000 7fbdeea00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbde000e970 tx=0x7fbde000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.486+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000cbd0 con 0x7fbdf0106570 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.486+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf01a2200 con 0x7fbdf0106570 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.486+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde000cd70 con 0x7fbdf0106570 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.486+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0005570 con 0x7fbdf0106570 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.486+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf01a9b20 con 0x7fbdf0106570 2024-05-27T06:54:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.487+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf00512b0 con 0x7fbdf0106570 2024-05-27T06:54:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.490+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbde0004750 con 0x7fbdf0106570 2024-05-27T06:54:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.492+0000 7fbde7400700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 0x7fbdd40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.492+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbde00e2140 con 0x7fbdf0106570 2024-05-27T06:54:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.492+0000 7fbdee000700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 0x7fbdd40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.493+0000 7fbdee000700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 0x7fbdd40b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b650 tx=0x7fbddc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.499+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbde0054df0 con 0x7fbdf0106570 2024-05-27T06:54:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.797+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdf019e3a0 con 0x7fbdf0106570 2024-05-27T06:54:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.799+0000 7fbde7400700 1 -- 172.21.15.16:0/4027366398 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fbde00aa260 con 0x7fbdf0106570 2024-05-27T06:54:14.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:14.792 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.802+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 msgr2=0x7fbdd40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.802+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 0x7fbdd40b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b650 tx=0x7fbddc005fb0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.802+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 msgr2=0x7fbdf019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.802+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbde000e970 tx=0x7fbde000ec80 comp rx=0 tx=0).stop 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 shutdown_connections 2024-05-27T06:54:14.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fbdd40b2900 0x7fbdd40b4e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbdf010c5b0 0x7fbdf01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdf0106570 0x7fbdf019d040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 --2- 172.21.15.16:0/4027366398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf0106ec0 0x7fbdf019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 >> 172.21.15.16:0/4027366398 conn(0x7fbdf0102070 msgr2=0x7fbdf01049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 shutdown_connections 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:14.803+0000 7fbdf5a00700 1 -- 172.21.15.16:0/4027366398 wait complete. 2024-05-27T06:54:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-05-27T06:54:14.980 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.16:6836/3811821671,v1:172.21.15.16:6837/3811821671] up:boot 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: fsmap 4 up:standby 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:14.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:14.982 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:14.982 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: Deploying daemon mds.e on smithi069 2024-05-27T06:54:14.982 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:14 smithi069 ceph-mon[35276]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.16:6836/3811821671,v1:172.21.15.16:6837/3811821671] up:boot 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: fsmap 4 up:standby 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:15.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:15.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:15.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: Deploying daemon mds.e on smithi069 2024-05-27T06:54:15.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:14 smithi164 ceph-mon[35327]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:15.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.16:6836/3811821671,v1:172.21.15.16:6837/3811821671] up:boot 2024-05-27T06:54:15.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: fsmap 4 up:standby 2024-05-27T06:54:15.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T06:54:15.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: Deploying daemon mds.e on smithi069 2024-05-27T06:54:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:14 smithi016 ceph-mon[31415]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:16.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:15 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4027366398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:16.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:15 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4027366398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:16.115 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:15 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4027366398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:16 smithi069 ceph-mon[35276]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:16 smithi164 ceph-mon[35327]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:16 smithi016 ceph-mon[31415]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:17.270 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:16 smithi069 systemd[1]: Starting Ceph mds.e for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:17.582 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 podman[64845]: 2024-05-27 06:54:17.189477946 +0000 UTC m=+0.043929393 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:17.582 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 podman[64845]: 2024-05-27 06:54:17.319571671 +0000 UTC m=+0.174023112 container create cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T06:54:17.582 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 podman[64845]: 2024-05-27 06:54:17.589679411 +0000 UTC m=+0.444130903 container init cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T06:54:17.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:17.870 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 podman[64845]: 2024-05-27 06:54:17.599556918 +0000 UTC m=+0.454008370 container start cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T06:54:17.870 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 bash[64845]: cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 2024-05-27T06:54:17.870 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 systemd[1]: Started Ceph mds.e for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:17.870 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 ceph-mds[64865]: -- 172.21.15.69:0/4012313212 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56300bb4ba40 con 0x56300bb93600 2024-05-27T06:54:17.870 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 06:54:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e[64861]: starting mds.e at 2024-05-27T06:54:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:19.050 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:boot 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: fsmap 5 up:standby 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: Deploying daemon mds.f on smithi164 2024-05-27T06:54:19.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:18 smithi164 ceph-mon[35327]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:19.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:boot 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: fsmap 5 up:standby 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:19.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: Deploying daemon mds.f on smithi164 2024-05-27T06:54:19.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:18 smithi016 ceph-mon[31415]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:boot 2024-05-27T06:54:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: fsmap 5 up:standby 2024-05-27T06:54:19.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: Deploying daemon mds.f on smithi164 2024-05-27T06:54:19.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:18 smithi069 ceph-mon[35276]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:20.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 -- 172.21.15.16:0/805565898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe0107de0 msgr2=0x7fabe01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 --2- 172.21.15.16:0/805565898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe0107de0 0x7fabe01081f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0099c0 tx=0x7fabcc009cd0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 -- 172.21.15.16:0/805565898 shutdown_connections 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 --2- 172.21.15.16:0/805565898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 0x7fabe0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 --2- 172.21.15.16:0/805565898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe01034b0 0x7fabe01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 --2- 172.21.15.16:0/805565898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe0107de0 0x7fabe01081f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 -- 172.21.15.16:0/805565898 >> 172.21.15.16:0/805565898 conn(0x7fabe00fede0 msgr2=0x7fabe0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 -- 172.21.15.16:0/805565898 shutdown_connections 2024-05-27T06:54:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.676+0000 7fabe7a00700 1 -- 172.21.15.16:0/805565898 wait complete. 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 Processor -- start 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 -- start start 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe0107de0 0x7fabe00ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 0x7fabe00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01131f0 con 0x7fabe01034b0 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0113070 con 0x7fabe0108730 2024-05-27T06:54:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.678+0000 7fabe7a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0113370 con 0x7fabe0107de0 2024-05-27T06:54:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50188/0 (socket says 172.21.15.16:50188) 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 -- 172.21.15.16:0/2998769004 learned_addr learned my addr 172.21.15.16:0/2998769004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabdfe00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe0107de0 0x7fabe00ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 0x7fabe00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 -- 172.21.15.16:0/2998769004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe0107de0 msgr2=0x7fabe00ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe0107de0 0x7fabe00ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 -- 172.21.15.16:0/2998769004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 msgr2=0x7fabe00ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 0x7fabe00ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.679+0000 7fabe5000700 1 -- 172.21.15.16:0/2998769004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe01a90d0 con 0x7fabe01034b0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabe5000700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d980 tx=0x7fabd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4016070 con 0x7fabe01034b0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd4004830 con 0x7fabe01034b0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4005440 con 0x7fabe01034b0 2024-05-27T06:54:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a93c0 con 0x7fabe01034b0 2024-05-27T06:54:20.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.680+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a9920 con 0x7fabe01034b0 2024-05-27T06:54:20.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.681+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe00512b0 con 0x7fabe01034b0 2024-05-27T06:54:20.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.684+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fabd40055e0 con 0x7fabe01034b0 2024-05-27T06:54:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.686+0000 7fabdd600700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 0x7fabc80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.686+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fabd40e2090 con 0x7fabe01034b0 2024-05-27T06:54:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.686+0000 7fabdfe00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 0x7fabc80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.687+0000 7fabdfe00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 0x7fabc80b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fabcc000c00 tx=0x7fabcc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:20.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.691+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabd401f020 con 0x7fabe01034b0 2024-05-27T06:54:20.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.979+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe002e5e0 con 0x7fabe01034b0 2024-05-27T06:54:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.982+0000 7fabdd600700 1 -- 172.21.15.16:0/2998769004 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7fabd40aa1b0 con 0x7fabe01034b0 2024-05-27T06:54:20.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:20.975 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T06:54:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.985+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 msgr2=0x7fabc80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.985+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 0x7fabc80b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fabcc000c00 tx=0x7fabcc01a040 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.985+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 msgr2=0x7fabe00ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.985+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d980 tx=0x7fabd400dc90 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 shutdown_connections 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fabc80b2900 0x7fabc80b4e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabe0108730 0x7fabe00ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe01034b0 0x7fabe00ff2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 --2- 172.21.15.16:0/2998769004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabe0107de0 0x7fabe00ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 >> 172.21.15.16:0/2998769004 conn(0x7fabe00fede0 msgr2=0x7fabe0075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 shutdown_connections 2024-05-27T06:54:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:20.986+0000 7fabe7a00700 1 -- 172.21.15.16:0/2998769004 wait complete. 2024-05-27T06:54:20.980 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-05-27T06:54:21.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:21 smithi164 ceph-mon[35327]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:21.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:21 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2998769004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:21 smithi016 ceph-mon[31415]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:21 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2998769004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:21 smithi069 ceph-mon[35276]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:21 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2998769004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:21.721 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 podman[64705]: 2024-05-27 06:54:21.480869636 +0000 UTC m=+0.192860493 container create a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T06:54:22.082 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 podman[64705]: 2024-05-27 06:54:21.777018302 +0000 UTC m=+0.489009232 container init a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T06:54:22.082 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 podman[64705]: 2024-05-27 06:54:21.787416553 +0000 UTC m=+0.499407446 container start a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T06:54:22.082 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 bash[64705]: a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 2024-05-27T06:54:22.082 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 systemd[1]: Started Ceph mds.f for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:22.082 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 ceph-mds[64731]: -- 172.21.15.164:0/3297349982 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55655fd69a40 con 0x55655fdb1a80 2024-05-27T06:54:22.083 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 06:54:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f[64722]: starting mds.f at 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: Deploying daemon mds.c on smithi164 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:23.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6834/2731773245,v1:172.21.15.164:6835/2731773245] up:boot 2024-05-27T06:54:23.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: fsmap 6 up:standby 2024-05-27T06:54:23.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:22 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: Deploying daemon mds.c on smithi164 2024-05-27T06:54:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:23.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6834/2731773245,v1:172.21.15.164:6835/2731773245] up:boot 2024-05-27T06:54:23.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: fsmap 6 up:standby 2024-05-27T06:54:23.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:22 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:23.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: Deploying daemon mds.c on smithi164 2024-05-27T06:54:23.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:23.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6834/2731773245,v1:172.21.15.164:6835/2731773245] up:boot 2024-05-27T06:54:23.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: fsmap 6 up:standby 2024-05-27T06:54:23.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:22 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T06:54:23.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:25.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:25 smithi164 ceph-mon[35327]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:25.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:25 smithi016 ceph-mon[31415]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:25 smithi069 ceph-mon[35276]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:25.751 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:25 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:26.035 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:25 smithi164 podman[65014]: 2024-05-27 06:54:25.663181407 +0000 UTC m=+0.033370252 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:26.036 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:25 smithi164 podman[65014]: 2024-05-27 06:54:25.771566099 +0000 UTC m=+0.141755005 container create 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-27T06:54:26.036 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 podman[65014]: 2024-05-27 06:54:26.041937426 +0000 UTC m=+0.412126379 container init 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T06:54:26.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 podman[65014]: 2024-05-27 06:54:26.054848113 +0000 UTC m=+0.425036986 container start 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T06:54:26.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 bash[65014]: 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b 2024-05-27T06:54:26.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:26.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 ceph-mds[65033]: -- 172.21.15.164:0/1128546316 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558516037a40 con 0x55851607f600 2024-05-27T06:54:26.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 06:54:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: starting mds.c at 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 -- 172.21.15.16:0/4146567235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c075d20 msgr2=0x7fd15c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 --2- 172.21.15.16:0/4146567235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c075d20 0x7fd15c076130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009990 tx=0x7fd14c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 -- 172.21.15.16:0/4146567235 shutdown_connections 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 --2- 172.21.15.16:0/4146567235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 0x7fd15c112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 --2- 172.21.15.16:0/4146567235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c075d20 0x7fd15c076130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 --2- 172.21.15.16:0/4146567235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c076670 0x7fd15c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 -- 172.21.15.16:0/4146567235 >> 172.21.15.16:0/4146567235 conn(0x7fd15c100030 msgr2=0x7fd15c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 -- 172.21.15.16:0/4146567235 shutdown_connections 2024-05-27T06:54:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.866+0000 7fd162600700 1 -- 172.21.15.16:0/4146567235 wait complete. 2024-05-27T06:54:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.868+0000 7fd162600700 1 Processor -- start 2024-05-27T06:54:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.868+0000 7fd162600700 1 -- start start 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c075d20 0x7fd15c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 0x7fd15c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c116530 con 0x7fd15c076670 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1163b0 con 0x7fd15c110b90 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd162600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1166b0 con 0x7fd15c075d20 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd15aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd15aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39762/0 (socket says 172.21.15.16:39762) 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd15aa00700 1 -- 172.21.15.16:0/3303429424 learned_addr learned my addr 172.21.15.16:0/3303429424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd15b400700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c075d20 0x7fd15c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.869+0000 7fd15be00700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 0x7fd15c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 -- 172.21.15.16:0/3303429424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c075d20 msgr2=0x7fd15c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c075d20 0x7fd15c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 -- 172.21.15.16:0/3303429424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 msgr2=0x7fd15c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 0x7fd15c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 -- 172.21.15.16:0/3303429424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c1a1dd0 con 0x7fd15c076670 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd15aa00700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c920 tx=0x7fd14800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148007b00 con 0x7fd15c076670 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd148004cc0 con 0x7fd15c076670 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c1a20c0 con 0x7fd15c076670 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1480054f0 con 0x7fd15c076670 2024-05-27T06:54:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.870+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1a99e0 con 0x7fd15c076670 2024-05-27T06:54:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.872+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c0512b0 con 0x7fd15c076670 2024-05-27T06:54:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.875+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd148007ca0 con 0x7fd15c076670 2024-05-27T06:54:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.878+0000 7fd153e00700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 0x7fd1400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.878+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd1480e1e60 con 0x7fd15c076670 2024-05-27T06:54:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.878+0000 7fd15b400700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 0x7fd1400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.879+0000 7fd15b400700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 0x7fd1400b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006010 tx=0x7fd14c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:26.880+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd148014030 con 0x7fd15c076670 2024-05-27T06:54:27.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.169+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd15c19e260 con 0x7fd15c076670 2024-05-27T06:54:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.170+0000 7fd153e00700 1 -- 172.21.15.16:0/3303429424 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7fd1480a9f80 con 0x7fd15c076670 2024-05-27T06:54:27.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:27.163 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T06:54:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.173+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 msgr2=0x7fd1400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.173+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 0x7fd1400b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006010 tx=0x7fd14c00b540 comp rx=0 tx=0).stop 2024-05-27T06:54:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.173+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 msgr2=0x7fd15c19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.173+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c920 tx=0x7fd14800cce0 comp rx=0 tx=0).stop 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.173+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 shutdown_connections 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd1400b2880 0x7fd1400b4d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c110b90 0x7fd15c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd15c075d20 0x7fd15c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 --2- 172.21.15.16:0/3303429424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c076670 0x7fd15c19d4a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 >> 172.21.15.16:0/3303429424 conn(0x7fd15c100030 msgr2=0x7fd15c100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 shutdown_connections 2024-05-27T06:54:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:27.174+0000 7fd162600700 1 -- 172.21.15.16:0/3303429424 wait complete. 2024-05-27T06:54:27.168 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-05-27T06:54:27.528 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: Deploying daemon mds.h on smithi069 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/162118671,v1:172.21.15.164:6837/162118671] up:boot 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: fsmap 7 up:standby 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T06:54:27.529 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:27 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3303429424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: Deploying daemon mds.h on smithi069 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/162118671,v1:172.21.15.164:6837/162118671] up:boot 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: fsmap 7 up:standby 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T06:54:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:27 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3303429424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:27.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: Deploying daemon mds.h on smithi069 2024-05-27T06:54:27.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/162118671,v1:172.21.15.164:6837/162118671] up:boot 2024-05-27T06:54:27.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: fsmap 7 up:standby 2024-05-27T06:54:27.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T06:54:27.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:27 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3303429424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:29 smithi164 ceph-mon[35327]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:29 smithi016 ceph-mon[31415]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:29 smithi069 ceph-mon[35276]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:29.929 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:29 smithi069 systemd[1]: Starting Ceph mds.h for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:30.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:30.181 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 podman[65151]: 2024-05-27 06:54:29.920373705 +0000 UTC m=+0.034585348 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:30.181 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 podman[65151]: 2024-05-27 06:54:30.033549741 +0000 UTC m=+0.147761420 container create c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T06:54:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 podman[65151]: 2024-05-27 06:54:30.294630935 +0000 UTC m=+0.408842594 container init c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 podman[65151]: 2024-05-27 06:54:30.303713636 +0000 UTC m=+0.417925349 container start c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 bash[65151]: c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 systemd[1]: Started Ceph mds.h for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 ceph-mds[65177]: -- 172.21.15.69:0/873958501 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55808d07ba40 con 0x55808d0c3a80 2024-05-27T06:54:30.620 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 06:54:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h[65171]: starting mds.h at 2024-05-27T06:54:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:31 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:31.563 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:31.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:31 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:31.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:31 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:32.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:32 smithi016 ceph-mon[31415]: Deploying daemon mds.i on smithi164 2024-05-27T06:54:32.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:32 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:boot 2024-05-27T06:54:32.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:32 smithi016 ceph-mon[31415]: fsmap 8 up:standby 2024-05-27T06:54:32.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:32 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T06:54:32.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:32 smithi164 ceph-mon[35327]: Deploying daemon mds.i on smithi164 2024-05-27T06:54:32.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:32 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:boot 2024-05-27T06:54:32.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:32 smithi164 ceph-mon[35327]: fsmap 8 up:standby 2024-05-27T06:54:32.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:32 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T06:54:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:32 smithi069 ceph-mon[35276]: Deploying daemon mds.i on smithi164 2024-05-27T06:54:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:32 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:boot 2024-05-27T06:54:32.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:32 smithi069 ceph-mon[35276]: fsmap 8 up:standby 2024-05-27T06:54:32.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:32 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 -- 172.21.15.16:0/3877678621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30110c60 msgr2=0x7f2c30113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3877678621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30110c60 0x7f2c30113080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b0a0 tx=0x7f2c2800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 -- 172.21.15.16:0/3877678621 shutdown_connections 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3877678621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30110c60 0x7f2c30113080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3877678621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c301030b0 0x7f2c301034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3877678621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c30103a00 0x7f2c30110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 -- 172.21.15.16:0/3877678621 >> 172.21.15.16:0/3877678621 conn(0x7f2c300fede0 msgr2=0x7f2c30101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 -- 172.21.15.16:0/3877678621 shutdown_connections 2024-05-27T06:54:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.929+0000 7f2c35a00700 1 -- 172.21.15.16:0/3877678621 wait complete. 2024-05-27T06:54:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.930+0000 7f2c35a00700 1 Processor -- start 2024-05-27T06:54:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.930+0000 7f2c35a00700 1 -- start start 2024-05-27T06:54:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c301030b0 0x7f2c3019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c30110c60 0x7f2c301a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30116560 con 0x7f2c30103a00 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301163e0 con 0x7f2c30110c60 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c35a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301166e0 con 0x7f2c301030b0 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c2e000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c2e000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52372/0 (socket says 172.21.15.16:52372) 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c2e000700 1 -- 172.21.15.16:0/3698467790 learned_addr learned my addr 172.21.15.16:0/3698467790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.931+0000 7f2c2ea00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c301030b0 0x7f2c3019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2f400700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c30110c60 0x7f2c301a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 -- 172.21.15.16:0/3698467790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c301030b0 msgr2=0x7f2c3019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c301030b0 0x7f2c3019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 -- 172.21.15.16:0/3698467790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c30110c60 msgr2=0x7f2c301a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c30110c60 0x7f2c301a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 -- 172.21.15.16:0/3698467790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c301a1de0 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c2e000700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c8f0 tx=0x7f2c1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.932+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c007b00 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.933+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c301a20d0 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.933+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c1c007ca0 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.933+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c018660 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.933+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c301a99f0 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.934+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c300512b0 con 0x7f2c30103a00 2024-05-27T06:54:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.937+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c1c00f450 con 0x7f2c30103a00 2024-05-27T06:54:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.939+0000 7f2c27400700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 0x7f2c140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.939+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2c1c0e1e50 con 0x7f2c30103a00 2024-05-27T06:54:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.939+0000 7f2c2ea00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 0x7f2c140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.940+0000 7f2c2ea00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 0x7f2c140b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005950 tx=0x7f2c200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:32.941+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c1c014030 con 0x7f2c30103a00 2024-05-27T06:54:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.233+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c3019e270 con 0x7f2c30103a00 2024-05-27T06:54:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.234+0000 7f2c27400700 1 -- 172.21.15.16:0/3698467790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f2c1c0aa000 con 0x7f2c30103a00 2024-05-27T06:54:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:33.231 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 msgr2=0x7f2c140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 0x7f2c140b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005950 tx=0x7f2c200058e0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 msgr2=0x7f2c3019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c8f0 tx=0x7f2c1c00ccb0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 shutdown_connections 2024-05-27T06:54:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2c140b2880 0x7f2c140b4d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c30110c60 0x7f2c301a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c301030b0 0x7f2c3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 --2- 172.21.15.16:0/3698467790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30103a00 0x7f2c3019d4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.238+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 >> 172.21.15.16:0/3698467790 conn(0x7f2c300fede0 msgr2=0x7f2c30112770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.239+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 shutdown_connections 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:33.239+0000 7f2c35a00700 1 -- 172.21.15.16:0/3698467790 wait complete. 2024-05-27T06:54:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-05-27T06:54:33.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:33 smithi164 ceph-mon[35327]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:33.491 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:33 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3698467790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:33 smithi016 ceph-mon[31415]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:33 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3698467790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:33 smithi069 ceph-mon[35276]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:33 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3698467790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:34.082 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:33 smithi164 systemd[1]: Starting Ceph mds.i for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:34.493 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 podman[65315]: 2024-05-27 06:54:34.078766043 +0000 UTC m=+0.032725871 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:34.493 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 podman[65315]: 2024-05-27 06:54:34.187215853 +0000 UTC m=+0.141175715 container create 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T06:54:34.493 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 podman[65315]: 2024-05-27 06:54:34.499856208 +0000 UTC m=+0.453816077 container init 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T06:54:34.833 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 podman[65315]: 2024-05-27 06:54:34.510018251 +0000 UTC m=+0.463978083 container start 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T06:54:34.833 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 bash[65315]: 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 2024-05-27T06:54:34.833 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 systemd[1]: Started Ceph mds.i for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:34.833 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 ceph-mds[65335]: -- 172.21.15.164:0/1051474100 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560433fc1a40 con 0x560434009180 2024-05-27T06:54:34.833 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 06:54:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i[65331]: starting mds.i at 2024-05-27T06:54:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:35.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:35 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:35 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:35.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:35 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:36.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:36 smithi164 ceph-mon[35327]: Deploying daemon mds.g on smithi016 2024-05-27T06:54:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:36 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:boot 2024-05-27T06:54:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:36 smithi164 ceph-mon[35327]: fsmap 9 up:standby 2024-05-27T06:54:36.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:36 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T06:54:36.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:36 smithi016 ceph-mon[31415]: Deploying daemon mds.g on smithi016 2024-05-27T06:54:36.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:36 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:boot 2024-05-27T06:54:36.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:36 smithi016 ceph-mon[31415]: fsmap 9 up:standby 2024-05-27T06:54:36.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:36 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T06:54:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:36 smithi069 ceph-mon[35276]: Deploying daemon mds.g on smithi016 2024-05-27T06:54:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:36 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:boot 2024-05-27T06:54:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:36 smithi069 ceph-mon[35276]: fsmap 9 up:standby 2024-05-27T06:54:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:36 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T06:54:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:37.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:37 smithi016 ceph-mon[31415]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:37 smithi164 ceph-mon[35327]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:37 smithi069 ceph-mon[35276]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:39.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:39 smithi016 ceph-mon[31415]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:39.514 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:39 smithi016 podman[71254]: 2024-05-27 06:54:39.342743409 +0000 UTC m=+0.170038998 container create 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T06:54:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:39 smithi164 ceph-mon[35327]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:39 smithi069 ceph-mon[35276]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:39.857 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:39 smithi016 podman[71254]: 2024-05-27 06:54:39.732242506 +0000 UTC m=+0.559538118 container init 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T06:54:39.857 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:39 smithi016 podman[71254]: 2024-05-27 06:54:39.743487016 +0000 UTC m=+0.570782608 container start 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T06:54:40.110 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:39 smithi016 ceph-mds[71291]: -- 172.21.15.16:0/1077981142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649834efa40 con 0x564983537180 2024-05-27T06:54:40.110 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g[71287]: starting mds.g at 2024-05-27T06:54:40.110 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:40 smithi016 bash[71254]: 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 2024-05-27T06:54:40.111 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 06:54:40 smithi016 systemd[1]: Started Ceph mds.g for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 -- 172.21.15.16:0/2817297147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798108f10 msgr2=0x7f9798109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 --2- 172.21.15.16:0/2817297147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798108f10 0x7f9798109320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97840099c0 tx=0x7f9784009cd0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 -- 172.21.15.16:0/2817297147 shutdown_connections 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 --2- 172.21.15.16:0/2817297147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9798109860 0x7f97981111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 --2- 172.21.15.16:0/2817297147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97981045e0 0x7f97981049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 --2- 172.21.15.16:0/2817297147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798108f10 0x7f9798109320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.442+0000 7f979e000700 1 -- 172.21.15.16:0/2817297147 >> 172.21.15.16:0/2817297147 conn(0x7f9798100030 msgr2=0x7f9798102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.443+0000 7f979e000700 1 -- 172.21.15.16:0/2817297147 shutdown_connections 2024-05-27T06:54:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.443+0000 7f979e000700 1 -- 172.21.15.16:0/2817297147 wait complete. 2024-05-27T06:54:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.444+0000 7f979e000700 1 Processor -- start 2024-05-27T06:54:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.444+0000 7f979e000700 1 -- start start 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97981045e0 0x7f979819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9798108f10 0x7f979819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9798114370 con 0x7f9798109860 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97981141f0 con 0x7f97981045e0 2024-05-27T06:54:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f979e000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97981144f0 con 0x7f9798108f10 2024-05-27T06:54:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.445+0000 7f9797e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52406/0 (socket says 172.21.15.16:52406) 2024-05-27T06:54:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 -- 172.21.15.16:0/930376493 learned_addr learned my addr 172.21.15.16:0/930376493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797400700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97981045e0 0x7f979819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9796a00700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9798108f10 0x7f979819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 -- 172.21.15.16:0/930376493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9798108f10 msgr2=0x7f979819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9798108f10 0x7f979819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 -- 172.21.15.16:0/930376493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97981045e0 msgr2=0x7f979819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97981045e0 0x7f979819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.446+0000 7f9797e00700 1 -- 172.21.15.16:0/930376493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97981a1d20 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f9797e00700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9790007d70 tx=0x7f979000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790016020 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97981a2010 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979000ac90 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97900115e0 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.447+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a24d0 con 0x7f9798109860 2024-05-27T06:54:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.448+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97980512b0 con 0x7f9798109860 2024-05-27T06:54:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.452+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9790019020 con 0x7f9798109860 2024-05-27T06:54:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.453+0000 7f978fe00700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 0x7f977c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.453+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f97900e22a0 con 0x7f9798109860 2024-05-27T06:54:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.453+0000 7f9797400700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 0x7f977c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.454+0000 7f9797400700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 0x7f977c0b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f978800ac50 tx=0x7f978800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.455+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9790020020 con 0x7f9798109860 2024-05-27T06:54:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.739+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f979819e260 con 0x7f9798109860 2024-05-27T06:54:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.740+0000 7f978fe00700 1 -- 172.21.15.16:0/930376493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f97900aa3c0 con 0x7f9798109860 2024-05-27T06:54:40.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:40.733 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 msgr2=0x7f977c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 0x7f977c0b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f978800ac50 tx=0x7f978800a380 comp rx=0 tx=0).stop 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 msgr2=0x7f97981a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9790007d70 tx=0x7f979000a340 comp rx=0 tx=0).stop 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 shutdown_connections 2024-05-27T06:54:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f977c0b2880 0x7f977c0b4d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9798109860 0x7f97981a1730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97981045e0 0x7f979819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 --2- 172.21.15.16:0/930376493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9798108f10 0x7f979819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 >> 172.21.15.16:0/930376493 conn(0x7f9798100030 msgr2=0x7f979810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 shutdown_connections 2024-05-27T06:54:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:40.743+0000 7f979e000700 1 -- 172.21.15.16:0/930376493 wait complete. 2024-05-27T06:54:40.738 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-05-27T06:54:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:41.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: Deploying daemon mds.a on smithi016 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.16:6838/1261534836,v1:172.21.15.16:6839/1261534836] up:boot 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: fsmap 10 up:standby 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T06:54:41.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:41 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/930376493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:41.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: Deploying daemon mds.a on smithi016 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.16:6838/1261534836,v1:172.21.15.16:6839/1261534836] up:boot 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: fsmap 10 up:standby 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T06:54:41.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:41 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/930376493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: Deploying daemon mds.a on smithi016 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.16:6838/1261534836,v1:172.21.15.16:6839/1261534836] up:boot 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: fsmap 10 up:standby 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T06:54:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:41 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/930376493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:43 smithi016 ceph-mon[31415]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:43 smithi164 ceph-mon[35327]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:43.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:43 smithi069 ceph-mon[35276]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:44.321 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:43 smithi016 systemd[1]: Starting Ceph mds.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:44.611 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 podman[71650]: 2024-05-27 06:54:44.232823116 +0000 UTC m=+0.034855742 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:44.611 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 podman[71650]: 2024-05-27 06:54:44.436137864 +0000 UTC m=+0.238170527 container create d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T06:54:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:45.083 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 podman[71650]: 2024-05-27 06:54:44.715535843 +0000 UTC m=+0.517568566 container init d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T06:54:45.083 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 podman[71650]: 2024-05-27 06:54:44.725997471 +0000 UTC m=+0.528030151 container start d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-27T06:54:45.083 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 bash[71650]: d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 2024-05-27T06:54:45.083 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 systemd[1]: Started Ceph mds.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:45.083 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 ceph-mds[71688]: -- 172.21.15.16:0/2912327431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8d418ba40 con 0x55d8d41d3180 2024-05-27T06:54:45.084 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 06:54:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a[71684]: starting mds.a at 2024-05-27T06:54:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:45 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:45.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:45 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T06:54:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:45 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:46.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:46 smithi016 ceph-mon[31415]: Deploying daemon mds.b on smithi069 2024-05-27T06:54:46.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:46 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.16:6840/1178863608,v1:172.21.15.16:6841/1178863608] up:boot 2024-05-27T06:54:46.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:46 smithi016 ceph-mon[31415]: fsmap 11 up:standby 2024-05-27T06:54:46.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:46 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T06:54:46.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:46 smithi069 ceph-mon[35276]: Deploying daemon mds.b on smithi069 2024-05-27T06:54:46.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:46 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.16:6840/1178863608,v1:172.21.15.16:6841/1178863608] up:boot 2024-05-27T06:54:46.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:46 smithi069 ceph-mon[35276]: fsmap 11 up:standby 2024-05-27T06:54:46.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:46 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T06:54:46.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:46 smithi164 ceph-mon[35327]: Deploying daemon mds.b on smithi069 2024-05-27T06:54:46.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:46 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.16:6840/1178863608,v1:172.21.15.16:6841/1178863608] up:boot 2024-05-27T06:54:46.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:46 smithi164 ceph-mon[35327]: fsmap 11 up:standby 2024-05-27T06:54:46.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:46 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T06:54:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 -- 172.21.15.16:0/2880332040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0108d20 msgr2=0x7f69e0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 --2- 172.21.15.16:0/2880332040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0108d20 0x7f69e0109130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0099c0 tx=0x7f69cc009cd0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 -- 172.21.15.16:0/2880332040 shutdown_connections 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 --2- 172.21.15.16:0/2880332040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0109670 0x7f69e0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 --2- 172.21.15.16:0/2880332040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 --2- 172.21.15.16:0/2880332040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0108d20 0x7f69e0109130 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 -- 172.21.15.16:0/2880332040 >> 172.21.15.16:0/2880332040 conn(0x7f69e0079d00 msgr2=0x7f69e007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 -- 172.21.15.16:0/2880332040 shutdown_connections 2024-05-27T06:54:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.396+0000 7f69dfe00700 1 -- 172.21.15.16:0/2880332040 wait complete. 2024-05-27T06:54:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.397+0000 7f69dfe00700 1 Processor -- start 2024-05-27T06:54:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 -- start start 2024-05-27T06:54:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0108d20 0x7f69e019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0114180 con 0x7f69e0109670 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0114000 con 0x7f69e01043f0 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69dfe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0114300 con 0x7f69e0108d20 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69df400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69df400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41390/0 (socket says 172.21.15.16:41390) 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.398+0000 7f69df400700 1 -- 172.21.15.16:0/1816902964 learned_addr learned my addr 172.21.15.16:0/1816902964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69dea00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:47.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69de000700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0108d20 0x7f69e019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 -- 172.21.15.16:0/1816902964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0108d20 msgr2=0x7f69e019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0108d20 0x7f69e019d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 -- 172.21.15.16:0/1816902964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 msgr2=0x7f69e019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 -- 172.21.15.16:0/1816902964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e01a1c40 con 0x7f69e0109670 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69dea00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.399+0000 7f69df400700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007ee0 tx=0x7f69d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.400+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800dbd0 con 0x7f69e0109670 2024-05-27T06:54:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.400+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e01a1f30 con 0x7f69e0109670 2024-05-27T06:54:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.400+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69d800dd70 con 0x7f69e0109670 2024-05-27T06:54:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.400+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80058d0 con 0x7f69e0109670 2024-05-27T06:54:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.400+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e01a9850 con 0x7f69e0109670 2024-05-27T06:54:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.401+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e01058a0 con 0x7f69e0109670 2024-05-27T06:54:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.404+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f69d801a020 con 0x7f69e0109670 2024-05-27T06:54:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.406+0000 7f69e4a00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 0x7f69c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.406+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f69d80e1cf0 con 0x7f69e0109670 2024-05-27T06:54:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.406+0000 7f69dea00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 0x7f69c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.407+0000 7f69dea00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 0x7f69c40b4d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69d000bc10 tx=0x7f69d000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.408+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69d8010070 con 0x7f69e0109670 2024-05-27T06:54:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:47 smithi164 ceph-mon[35327]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:47 smithi016 ceph-mon[31415]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:47.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:47 smithi069 ceph-mon[35276]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.697+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69e00512b0 con 0x7f69e0109670 2024-05-27T06:54:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.699+0000 7f69e4a00700 1 -- 172.21.15.16:0/1816902964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f69d80a9e10 con 0x7f69e0109670 2024-05-27T06:54:47.692 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:47.692 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 msgr2=0x7f69c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 0x7f69c40b4d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69d000bc10 tx=0x7f69d000b5f0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 msgr2=0x7f69e01a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007ee0 tx=0x7f69d800d3b0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 shutdown_connections 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69c40b2880 0x7f69c40b4d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0109670 0x7f69e01a15a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e01043f0 0x7f69e019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.702+0000 7f69dfe00700 1 --2- 172.21.15.16:0/1816902964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69e0108d20 0x7f69e019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.703+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 >> 172.21.15.16:0/1816902964 conn(0x7f69e0079d00 msgr2=0x7f69e0101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.703+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 shutdown_connections 2024-05-27T06:54:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:47.703+0000 7f69dfe00700 1 -- 172.21.15.16:0/1816902964 wait complete. 2024-05-27T06:54:47.697 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-05-27T06:54:48.240 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:47 smithi069 systemd[1]: Starting Ceph mds.b for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T06:54:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:48 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1816902964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:48.491 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 podman[65459]: 2024-05-27 06:54:48.201965198 +0000 UTC m=+0.031793705 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T06:54:48.492 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 podman[65459]: 2024-05-27 06:54:48.320823259 +0000 UTC m=+0.150651791 container create 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T06:54:48.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:48 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1816902964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:48.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:48 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1816902964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:48.745 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 podman[65459]: 2024-05-27 06:54:48.625267052 +0000 UTC m=+0.455095597 container init 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T06:54:48.745 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 podman[65459]: 2024-05-27 06:54:48.634615516 +0000 UTC m=+0.464444026 container start 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T06:54:48.745 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 bash[65459]: 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a 2024-05-27T06:54:48.746 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 systemd[1]: Started Ceph mds.b for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T06:54:48.746 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 ceph-mds[65479]: -- 172.21.15.69:0/2643704135 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f0eb89a40 con 0x556f0ebd1a80 2024-05-27T06:54:49.119 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:54:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: starting mds.b at 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:49 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:49 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:49 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:50.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:50 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:boot 2024-05-27T06:54:50.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:50 smithi069 ceph-mon[35276]: fsmap 12 up:standby 2024-05-27T06:54:50.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T06:54:50.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:50 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:50 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:boot 2024-05-27T06:54:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:50 smithi164 ceph-mon[35327]: fsmap 12 up:standby 2024-05-27T06:54:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T06:54:50.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:50 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:50 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:boot 2024-05-27T06:54:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:50 smithi016 ceph-mon[31415]: fsmap 12 up:standby 2024-05-27T06:54:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T06:54:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:50 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:50.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:54:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:51.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:51 smithi016 ceph-mon[31415]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:51.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:51 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:51.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:51 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:51 smithi164 ceph-mon[35327]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:51 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:51 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:51 smithi069 ceph-mon[35276]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:51 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:51 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:53 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:53 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:53 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 -- 172.21.15.16:0/1984775679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce1010e8d0 msgr2=0x7fce10110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 --2- 172.21.15.16:0/1984775679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce1010e8d0 0x7fce10110db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b0a0 tx=0x7fce0c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 -- 172.21.15.16:0/1984775679 shutdown_connections 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 --2- 172.21.15.16:0/1984775679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce1010e8d0 0x7fce10110db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 --2- 172.21.15.16:0/1984775679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce101054f0 0x7fce10105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 --2- 172.21.15.16:0/1984775679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce10105e40 0x7fce1010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.038+0000 7fce18800700 1 -- 172.21.15.16:0/1984775679 >> 172.21.15.16:0/1984775679 conn(0x7fce100fedb0 msgr2=0x7fce101011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.039+0000 7fce18800700 1 -- 172.21.15.16:0/1984775679 shutdown_connections 2024-05-27T06:54:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.039+0000 7fce18800700 1 -- 172.21.15.16:0/1984775679 wait complete. 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.040+0000 7fce18800700 1 Processor -- start 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.040+0000 7fce18800700 1 -- start start 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce10105e40 0x7fce1019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce1010e8d0 0x7fce101a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce10114330 con 0x7fce101054f0 2024-05-27T06:54:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce101141b0 con 0x7fce1010e8d0 2024-05-27T06:54:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce18800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce101144b0 con 0x7fce10105e40 2024-05-27T06:54:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce15e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce15e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce15e00700 1 -- 172.21.15.16:0/2418976242 learned_addr learned my addr 172.21.15.16:0/2418976242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce15400700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce10105e40 0x7fce1019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.041+0000 7fce16800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce1010e8d0 0x7fce101a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 -- 172.21.15.16:0/2418976242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce10105e40 msgr2=0x7fce1019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce10105e40 0x7fce1019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 -- 172.21.15.16:0/2418976242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce1010e8d0 msgr2=0x7fce101a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce1010e8d0 0x7fce101a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 -- 172.21.15.16:0/2418976242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce101a1d00 con 0x7fce101054f0 2024-05-27T06:54:54.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce15e00700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce0400d950 tx=0x7fce0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04016070 con 0x7fce101054f0 2024-05-27T06:54:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce04010460 con 0x7fce101054f0 2024-05-27T06:54:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.042+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce101a1ff0 con 0x7fce101054f0 2024-05-27T06:54:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.043+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0400f620 con 0x7fce101054f0 2024-05-27T06:54:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.043+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce101a2530 con 0x7fce101054f0 2024-05-27T06:54:54.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.044+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce100512b0 con 0x7fce101054f0 2024-05-27T06:54:54.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.049+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fce04019070 con 0x7fce101054f0 2024-05-27T06:54:54.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.050+0000 7fce0aa00700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 0x7fcdf80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:54.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.051+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fce040e19a0 con 0x7fce101054f0 2024-05-27T06:54:54.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.051+0000 7fce15400700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 0x7fcdf80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.052+0000 7fce15400700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 0x7fcdf80b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce00005fd0 tx=0x7fce0000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.052+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce040a9b60 con 0x7fce101054f0 2024-05-27T06:54:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.354+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce1019e240 con 0x7fce101054f0 2024-05-27T06:54:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.355+0000 7fce0aa00700 1 -- 172.21.15.16:0/2418976242 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7fce1019e240 con 0x7fce101054f0 2024-05-27T06:54:54.347 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:54.347 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-27T06:54:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 msgr2=0x7fcdf80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 0x7fcdf80b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce00005fd0 tx=0x7fce0000af20 comp rx=0 tx=0).stop 2024-05-27T06:54:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 msgr2=0x7fce1019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce0400d950 tx=0x7fce0400dd10 comp rx=0 tx=0).stop 2024-05-27T06:54:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 shutdown_connections 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdf80b2880 0x7fcdf80b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce1010e8d0 0x7fce101a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce101054f0 0x7fce1019cee0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 --2- 172.21.15.16:0/2418976242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce10105e40 0x7fce1019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.358+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 >> 172.21.15.16:0/2418976242 conn(0x7fce100fedb0 msgr2=0x7fce101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.359+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 shutdown_connections 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:54.359+0000 7fce18800700 1 -- 172.21.15.16:0/2418976242 wait complete. 2024-05-27T06:54:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-05-27T06:54:55.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-27T06:54:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:55 smithi164 ceph-mon[35327]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:55 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2418976242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:55.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:55 smithi016 ceph-mon[31415]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:55.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:55 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2418976242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:55 smithi069 ceph-mon[35276]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:55 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2418976242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:54:55.706 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:54:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:56 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:56 smithi016 ceph-mon[31415]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:56.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:56 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:56 smithi164 ceph-mon[35327]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:56.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:56 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:56.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:56 smithi069 ceph-mon[35276]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:57.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:57 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:57.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:57 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:54:57.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:57 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:57.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:57 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:57.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:57 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:57.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:57 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:54:57.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:57 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:57.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:57 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:57.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:57 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:54:57.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:57 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:54:57.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:57 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:54:57.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:57 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:54:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 -- 172.21.15.16:0/4018649041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f557410c5a0 msgr2=0x7f557410c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 --2- 172.21.15.16:0/4018649041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f557410c5a0 0x7f557410c9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f90 tx=0x7f556400b330 comp rx=0 tx=0).stop 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 -- 172.21.15.16:0/4018649041 shutdown_connections 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 --2- 172.21.15.16:0/4018649041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5574106eb0 0x7f5574113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 --2- 172.21.15.16:0/4018649041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f557410c5a0 0x7f557410c9b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 --2- 172.21.15.16:0/4018649041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106560 0x7f5574106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.248+0000 7f5579a00700 1 -- 172.21.15.16:0/4018649041 >> 172.21.15.16:0/4018649041 conn(0x7f5574102040 msgr2=0x7f5574104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.249+0000 7f5579a00700 1 -- 172.21.15.16:0/4018649041 shutdown_connections 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.249+0000 7f5579a00700 1 -- 172.21.15.16:0/4018649041 wait complete. 2024-05-27T06:54:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.250+0000 7f5579a00700 1 Processor -- start 2024-05-27T06:54:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.250+0000 7f5579a00700 1 -- start start 2024-05-27T06:54:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106eb0 0x7f557419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557410c5a0 0x7f55741a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55741165d0 con 0x7f557410c5a0 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5574116450 con 0x7f5574106eb0 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5579a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5574116750 con 0x7f5574106560 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5572a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5572a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:38504/0 (socket says 172.21.15.16:38504) 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5572a00700 1 -- 172.21.15.16:0/695529462 learned_addr learned my addr 172.21.15.16:0/695529462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:54:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5573400700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557410c5a0 0x7f55741a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.251+0000 7f5572000700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106eb0 0x7f557419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 -- 172.21.15.16:0/695529462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106eb0 msgr2=0x7f557419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106eb0 0x7f557419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 -- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557410c5a0 msgr2=0x7f55741a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557410c5a0 0x7f55741a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 -- 172.21.15.16:0/695529462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741a1e70 con 0x7f5574106560 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.252+0000 7f5572a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5564009790 tx=0x7f556400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.253+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564021070 con 0x7f5574106560 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.253+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5564007ac0 con 0x7f5574106560 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.253+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741a2100 con 0x7f5574106560 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.253+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564005040 con 0x7f5574106560 2024-05-27T06:54:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.253+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741a9a80 con 0x7f5574106560 2024-05-27T06:54:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.254+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55740512b0 con 0x7f5574106560 2024-05-27T06:54:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.259+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5564024070 con 0x7f5574106560 2024-05-27T06:54:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.261+0000 7f556b400700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 0x7f55580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:54:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.261+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f556400f020 con 0x7f5574106560 2024-05-27T06:54:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.261+0000 7f5572000700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 0x7f55580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:54:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.262+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f556402b020 con 0x7f5574106560 2024-05-27T06:54:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.262+0000 7f5572000700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 0x7f55580b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5574105570 tx=0x7f555c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:54:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.534+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f557402e5e0 con 0x7f5574106560 2024-05-27T06:54:58.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:58 smithi016 ceph-mon[31415]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.535+0000 7f556b400700 1 -- 172.21.15.16:0/695529462 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f55640ab6c0 con 0x7f5574106560 2024-05-27T06:54:58.527 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:54:58.527 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-05-27T06:54:58.527 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T06:54:58.527 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-05-27T06:54:58.528 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-05-27T06:54:58.528 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T06:54:58.528 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 msgr2=0x7f55580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 0x7f55580b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5574105570 tx=0x7f555c009260 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 msgr2=0x7f557419cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5564009790 tx=0x7f556400cc30 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 shutdown_connections 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f55580b2900 0x7f55580b4e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557410c5a0 0x7f55741a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5574106560 0x7f557419cfa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 --2- 172.21.15.16:0/695529462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5574106eb0 0x7f557419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:54:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.539+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 >> 172.21.15.16:0/695529462 conn(0x7f5574102040 msgr2=0x7f5574103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:54:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.540+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 shutdown_connections 2024-05-27T06:54:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:54:58.540+0000 7f5579a00700 1 -- 172.21.15.16:0/695529462 wait complete. 2024-05-27T06:54:58.726 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:58 smithi069 ceph-mon[35276]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:58.764 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:58 smithi164 ceph-mon[35327]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:54:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:54:59 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/695529462' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T06:54:59.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:54:59 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/695529462' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T06:54:59.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:54:59 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/695529462' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T06:55:00.144 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T06:55:00.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T06:55:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:01 smithi016 ceph-mon[31415]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:01 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:01 smithi069 ceph-mon[35276]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:01 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:01 smithi164 ceph-mon[35327]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:01.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:01 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 -- 172.21.15.16:0/1360093338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 msgr2=0x7f1b00105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 --2- 172.21.15.16:0/1360093338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b00105690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009990 tx=0x7f1af0009ca0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 -- 172.21.15.16:0/1360093338 shutdown_connections 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 --2- 172.21.15.16:0/1360093338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010e6f0 0x7f1b00110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 --2- 172.21.15.16:0/1360093338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b00105690 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 --2- 172.21.15.16:0/1360093338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b00105bd0 0x7f1b0010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 -- 172.21.15.16:0/1360093338 >> 172.21.15.16:0/1360093338 conn(0x7f1b00078ab0 msgr2=0x7f1b00078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 -- 172.21.15.16:0/1360093338 shutdown_connections 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.252+0000 7f1b04c00700 1 -- 172.21.15.16:0/1360093338 wait complete. 2024-05-27T06:55:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.254+0000 7f1b04c00700 1 Processor -- start 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:55:03.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.254+0000 7f1b04c00700 1 -- start start 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00105bd0 0x7f1b0019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010e6f0 0x7f1b0019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00114240 con 0x7f1b00105280 2024-05-27T06:55:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001140c0 con 0x7f1b00105bd0 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1b04c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001143c0 con 0x7f1b0010e6f0 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1aff400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1aff400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38676/0 (socket says 172.21.15.16:38676) 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1aff400700 1 -- 172.21.15.16:0/3453137103 learned_addr learned my addr 172.21.15.16:0/3453137103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1affe00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010e6f0 0x7f1b0019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.255+0000 7f1afea00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00105bd0 0x7f1b0019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 -- 172.21.15.16:0/3453137103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010e6f0 msgr2=0x7f1b0019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010e6f0 0x7f1b0019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 -- 172.21.15.16:0/3453137103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00105bd0 msgr2=0x7f1b0019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00105bd0 0x7f1b0019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 -- 172.21.15.16:0/3453137103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0019fbe0 con 0x7f1b00105280 2024-05-27T06:55:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.256+0000 7f1aff400700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000c00 tx=0x7f1af00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.257+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0026070 con 0x7f1b00105280 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.257+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af000fb30 con 0x7f1b00105280 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.257+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0019fe70 con 0x7f1b00105280 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.257+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0021bb0 con 0x7f1b00105280 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.257+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a77f0 con 0x7f1b00105280 2024-05-27T06:55:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.258+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0005330 con 0x7f1b00105280 2024-05-27T06:55:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.261+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1af0021410 con 0x7f1b00105280 2024-05-27T06:55:03.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.263+0000 7f1af7e00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 0x7f1ae40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:03.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.263+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1af00e2bb0 con 0x7f1b00105280 2024-05-27T06:55:03.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.263+0000 7f1afea00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 0x7f1ae40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:03.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.264+0000 7f1afea00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 0x7f1ae40b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b001026f0 tx=0x7f1aec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.265+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1af0013070 con 0x7f1b00105280 2024-05-27T06:55:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:55:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:55:03.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:03.503+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1ae0005620 con 0x7f1b00105280 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:55:03.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:55:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.259+0000 7f1af7e00700 1 -- 172.21.15.16:0/3453137103 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f1af00aad60 con 0x7f1b00105280 2024-05-27T06:55:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 msgr2=0x7f1ae40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 0x7f1ae40b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b001026f0 tx=0x7f1aec009190 comp rx=0 tx=0).stop 2024-05-27T06:55:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 msgr2=0x7f1b0019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000c00 tx=0x7f1af00057c0 comp rx=0 tx=0).stop 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 shutdown_connections 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ae40b2900 0x7f1ae40b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010e6f0 0x7f1b0019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00105280 0x7f1b0019ad10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3453137103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00105bd0 0x7f1b0019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.264+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 >> 172.21.15.16:0/3453137103 conn(0x7f1b00078ab0 msgr2=0x7f1b00100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.265+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 shutdown_connections 2024-05-27T06:55:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:04.265+0000 7f1b04c00700 1 -- 172.21.15.16:0/3453137103 wait complete. 2024-05-27T06:55:04.260 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-05-27T06:55:04.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:04 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T06:55:04.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:04 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T06:55:04.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:04 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T06:55:05.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-27T06:55:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:05 smithi016 ceph-mon[31415]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:05 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T06:55:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:05 smithi016 ceph-mon[31415]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T06:55:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:05 smithi164 ceph-mon[35327]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:05 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T06:55:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:05 smithi164 ceph-mon[35327]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T06:55:05.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:05 smithi069 ceph-mon[35276]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:05.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:05 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3453137103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T06:55:05.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:05 smithi069 ceph-mon[35276]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T06:55:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:06.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:06 smithi016 ceph-mon[31415]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T06:55:06.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:06 smithi164 ceph-mon[35327]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T06:55:06.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:06 smithi069 ceph-mon[35276]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T06:55:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:07 smithi016 ceph-mon[31415]: pgmap v378: 33 pgs: 6 creating+activating, 4 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:07 smithi016 ceph-mon[31415]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:07 smithi016 ceph-mon[31415]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T06:55:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:07 smithi164 ceph-mon[35327]: pgmap v378: 33 pgs: 6 creating+activating, 4 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:07 smithi164 ceph-mon[35327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:07 smithi164 ceph-mon[35327]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T06:55:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:07 smithi069 ceph-mon[35276]: pgmap v378: 33 pgs: 6 creating+activating, 4 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:07 smithi069 ceph-mon[35276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:07 smithi069 ceph-mon[35276]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T06:55:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 -- 172.21.15.16:0/1556263154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 msgr2=0x7f8848109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 --2- 172.21.15.16:0/1556263154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f8848109300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f88300099c0 tx=0x7f8830009cd0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 -- 172.21.15.16:0/1556263154 shutdown_connections 2024-05-27T06:55:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 --2- 172.21.15.16:0/1556263154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848109840 0x7f88481111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 --2- 172.21.15.16:0/1556263154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88481045c0 0x7f88481049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 --2- 172.21.15.16:0/1556263154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f8848109300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 -- 172.21.15.16:0/1556263154 >> 172.21.15.16:0/1556263154 conn(0x7f88480ffe90 msgr2=0x7f88481022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 -- 172.21.15.16:0/1556263154 shutdown_connections 2024-05-27T06:55:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.063+0000 7f884ca00700 1 -- 172.21.15.16:0/1556263154 wait complete. 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.064+0000 7f884ca00700 1 Processor -- start 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.064+0000 7f884ca00700 1 -- start start 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481045c0 0x7f88481a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8848109840 0x7f884819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8848114300 con 0x7f8848108ef0 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8848114180 con 0x7f8848109840 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f884ca00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8848114480 con 0x7f88481045c0 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f883d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f883d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38692/0 (socket says 172.21.15.16:38692) 2024-05-27T06:55:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.065+0000 7f883d400700 1 -- 172.21.15.16:0/2968873944 learned_addr learned my addr 172.21.15.16:0/2968873944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f8846800700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8848109840 0x7f884819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f8845e00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481045c0 0x7f88481a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 -- 172.21.15.16:0/2968873944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481045c0 msgr2=0x7f88481a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481045c0 0x7f88481a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 -- 172.21.15.16:0/2968873944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8848109840 msgr2=0x7f884819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8848109840 0x7f884819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 -- 172.21.15.16:0/2968873944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884819b170 con 0x7f8848108ef0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.066+0000 7f883d400700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f883000b580 tx=0x7f883000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.067+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830027070 con 0x7f8848108ef0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.067+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f883000fc20 con 0x7f8848108ef0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.067+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884819b400 con 0x7f8848108ef0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.067+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830017410 con 0x7f8848108ef0 2024-05-27T06:55:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.067+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884819b930 con 0x7f8848108ef0 2024-05-27T06:55:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.068+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88480512b0 con 0x7f8848108ef0 2024-05-27T06:55:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.071+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f883002a070 con 0x7f8848108ef0 2024-05-27T06:55:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.072+0000 7f883f400700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 0x7f882c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.073+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f88300e3200 con 0x7f8848108ef0 2024-05-27T06:55:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.073+0000 7f8845e00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 0x7f882c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.074+0000 7f8845e00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 0x7f882c0b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f884819bc40 tx=0x7f8838009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.076+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f884819b930 con 0x7f8848108ef0 2024-05-27T06:55:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:08.318+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f884802e5e0 con 0x7f8848108ef0 2024-05-27T06:55:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.317+0000 7f883f400700 1 -- 172.21.15.16:0/2968873944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f8830013070 con 0x7f8848108ef0 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 msgr2=0x7f882c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 0x7f882c0b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f884819bc40 tx=0x7f8838009040 comp rx=0 tx=0).stop 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 msgr2=0x7f88481a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f883000b580 tx=0x7f883000f740 comp rx=0 tx=0).stop 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 shutdown_connections 2024-05-27T06:55:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f882c0b2900 0x7f882c0b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8848109840 0x7f884819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481045c0 0x7f88481a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 --2- 172.21.15.16:0/2968873944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8848108ef0 0x7f88481a0e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 >> 172.21.15.16:0/2968873944 conn(0x7f88480ffe90 msgr2=0x7f884810a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.322+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 shutdown_connections 2024-05-27T06:55:09.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:09.323+0000 7f884ca00700 1 -- 172.21.15.16:0/2968873944 wait complete. 2024-05-27T06:55:09.318 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-05-27T06:55:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:09 smithi164 ceph-mon[35327]: pgmap v380: 33 pgs: 6 creating+activating, 4 creating+peering, 5 unknown, 18 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:09 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T06:55:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:09 smithi016 ceph-mon[31415]: pgmap v380: 33 pgs: 6 creating+activating, 4 creating+peering, 5 unknown, 18 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:09 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T06:55:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:09 smithi069 ceph-mon[35276]: pgmap v380: 33 pgs: 6 creating+activating, 4 creating+peering, 5 unknown, 18 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:09 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T06:55:10.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T06:55:10.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:10 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T06:55:10.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:10 smithi016 ceph-mon[31415]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T06:55:10.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:10 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T06:55:10.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:10 smithi069 ceph-mon[35276]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T06:55:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:10.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:10 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2968873944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T06:55:10.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:10 smithi164 ceph-mon[35327]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T06:55:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:11 smithi016 ceph-mon[31415]: pgmap v382: 97 pgs: 50 unknown, 14 creating+peering, 6 creating+activating, 27 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:11 smithi016 ceph-mon[31415]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T06:55:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:11 smithi069 ceph-mon[35276]: pgmap v382: 97 pgs: 50 unknown, 14 creating+peering, 6 creating+activating, 27 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:11 smithi069 ceph-mon[35276]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T06:55:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:11 smithi164 ceph-mon[35327]: pgmap v382: 97 pgs: 50 unknown, 14 creating+peering, 6 creating+activating, 27 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:11 smithi164 ceph-mon[35327]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T06:55:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:12 smithi069 ceph-mon[35276]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T06:55:12.794 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:12 smithi016 ceph-mon[31415]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T06:55:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:12 smithi164 ceph-mon[35327]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.205+0000 7fac60400700 1 -- 172.21.15.16:0/236932592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 msgr2=0x7fac581069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.205+0000 7fac60400700 1 --2- 172.21.15.16:0/236932592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac581069a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac44009990 tx=0x7fac44009ca0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 -- 172.21.15.16:0/236932592 shutdown_connections 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 --2- 172.21.15.16:0/236932592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 0x7fac58113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 --2- 172.21.15.16:0/236932592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 0x7fac5810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 --2- 172.21.15.16:0/236932592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac581069a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 -- 172.21.15.16:0/236932592 >> 172.21.15.16:0/236932592 conn(0x7fac581020b0 msgr2=0x7fac58104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 -- 172.21.15.16:0/236932592 shutdown_connections 2024-05-27T06:55:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.206+0000 7fac60400700 1 -- 172.21.15.16:0/236932592 wait complete. 2024-05-27T06:55:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.207+0000 7fac60400700 1 Processor -- start 2024-05-27T06:55:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.207+0000 7fac60400700 1 -- start start 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 0x7fac5819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 0x7fac581a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac581165b0 con 0x7fac58106590 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac58116430 con 0x7fac58106ee0 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac60400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac58116730 con 0x7fac5810c5d0 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac5da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac5da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56512/0 (socket says 172.21.15.16:56512) 2024-05-27T06:55:13.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.208+0000 7fac5da00700 1 -- 172.21.15.16:0/4258529695 learned_addr learned my addr 172.21.15.16:0/4258529695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5e400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 0x7fac581a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5d000700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 0x7fac5819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 -- 172.21.15.16:0/4258529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 msgr2=0x7fac581a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 0x7fac581a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 -- 172.21.15.16:0/4258529695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 msgr2=0x7fac5819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 0x7fac5819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 -- 172.21.15.16:0/4258529695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac581a1de0 con 0x7fac58106590 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.209+0000 7fac5da00700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00d980 tx=0x7fac4c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.210+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c017070 con 0x7fac58106590 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.210+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac4c010460 con 0x7fac58106590 2024-05-27T06:55:13.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.210+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00de60 con 0x7fac58106590 2024-05-27T06:55:13.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.210+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac581a20d0 con 0x7fac58106590 2024-05-27T06:55:13.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.210+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac581a25c0 con 0x7fac58106590 2024-05-27T06:55:13.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.211+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5810e990 con 0x7fac58106590 2024-05-27T06:55:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.214+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fac4c010610 con 0x7fac58106590 2024-05-27T06:55:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.215+0000 7fac56000700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 0x7fac400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.215+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7fac4c0e1b90 con 0x7fac58106590 2024-05-27T06:55:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.216+0000 7fac5d000700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 0x7fac400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.217+0000 7fac5d000700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 0x7fac400b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fac58103f10 tx=0x7fac44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.219+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac4c0a9a40 con 0x7fac58106590 2024-05-27T06:55:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:13.516+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fac580512b0 con 0x7fac58106590 2024-05-27T06:55:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:13 smithi069 ceph-mon[35276]: pgmap v385: 97 pgs: 30 unknown, 18 creating+peering, 49 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:13 smithi069 ceph-mon[35276]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:13 smithi164 ceph-mon[35327]: pgmap v385: 97 pgs: 30 unknown, 18 creating+peering, 49 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:13 smithi164 ceph-mon[35327]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:13 smithi016 ceph-mon[31415]: pgmap v385: 97 pgs: 30 unknown, 18 creating+peering, 49 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:13 smithi016 ceph-mon[31415]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.390+0000 7fac56000700 1 -- 172.21.15.16:0/4258529695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fac4c0a9340 con 0x7fac58106590 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.396+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 msgr2=0x7fac400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.396+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 0x7fac400b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fac58103f10 tx=0x7fac44005fb0 comp rx=0 tx=0).stop 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.396+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 msgr2=0x7fac5819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.396+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00d980 tx=0x7fac4c00dd40 comp rx=0 tx=0).stop 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 shutdown_connections 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fac400b2900 0x7fac400b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac5810c5d0 0x7fac581a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac58106590 0x7fac5819cfc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 --2- 172.21.15.16:0/4258529695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac58106ee0 0x7fac5819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 >> 172.21.15.16:0/4258529695 conn(0x7fac581020b0 msgr2=0x7fac58104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 shutdown_connections 2024-05-27T06:55:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:14.397+0000 7fac60400700 1 -- 172.21.15.16:0/4258529695 wait complete. 2024-05-27T06:55:14.393 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T06:55:14.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:14 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T06:55:14.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:14 smithi016 ceph-mon[31415]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T06:55:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:14 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T06:55:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:14 smithi164 ceph-mon[35327]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T06:55:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:14 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T06:55:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:14 smithi069 ceph-mon[35276]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T06:55:15.343 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-27T06:55:15.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-27T06:55:15.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:15 smithi016 ceph-mon[31415]: pgmap v386: 97 pgs: 12 unknown, 18 creating+peering, 67 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:15.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:15 smithi016 ceph-mon[31415]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T06:55:15.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:15 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T06:55:15.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:15 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T06:55:15.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:15 smithi016 ceph-mon[31415]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T06:55:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:15 smithi164 ceph-mon[35327]: pgmap v386: 97 pgs: 12 unknown, 18 creating+peering, 67 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:15 smithi164 ceph-mon[35327]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T06:55:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:15 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T06:55:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:15 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T06:55:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:15 smithi164 ceph-mon[35327]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T06:55:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:15 smithi069 ceph-mon[35276]: pgmap v386: 97 pgs: 12 unknown, 18 creating+peering, 67 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:15 smithi069 ceph-mon[35276]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T06:55:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:15 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4258529695' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T06:55:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:15 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T06:55:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:15 smithi069 ceph-mon[35276]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T06:55:16.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:16 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:active 2024-05-27T06:55:16.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:16 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:16.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:16 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:16 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:active 2024-05-27T06:55:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:16 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:16 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:16 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] up:active 2024-05-27T06:55:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:16 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:16 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:17.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:17 smithi016 ceph-mon[31415]: pgmap v388: 97 pgs: 97 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-27T06:55:17.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:17 smithi016 ceph-mon[31415]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T06:55:17.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:17 smithi016 ceph-mon[31415]: Cluster is now healthy 2024-05-27T06:55:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:17 smithi164 ceph-mon[35327]: pgmap v388: 97 pgs: 97 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-27T06:55:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:17 smithi164 ceph-mon[35327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T06:55:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:17 smithi164 ceph-mon[35327]: Cluster is now healthy 2024-05-27T06:55:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:17 smithi069 ceph-mon[35276]: pgmap v388: 97 pgs: 97 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-27T06:55:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:17 smithi069 ceph-mon[35276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T06:55:17.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:17 smithi069 ceph-mon[35276]: Cluster is now healthy 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.212+0000 7fc2e7400700 1 -- 172.21.15.16:0/1420138072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e01045e0 msgr2=0x7fc2e01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.212+0000 7fc2e7400700 1 --2- 172.21.15.16:0/1420138072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e01045e0 0x7fc2e01049f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40099c0 tx=0x7fc2d4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 -- 172.21.15.16:0/1420138072 shutdown_connections 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 --2- 172.21.15.16:0/1420138072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 0x7fc2e01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 --2- 172.21.15.16:0/1420138072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e01045e0 0x7fc2e01049f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 --2- 172.21.15.16:0/1420138072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e0108f10 0x7fc2e0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 -- 172.21.15.16:0/1420138072 >> 172.21.15.16:0/1420138072 conn(0x7fc2e0100030 msgr2=0x7fc2e0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 -- 172.21.15.16:0/1420138072 shutdown_connections 2024-05-27T06:55:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.213+0000 7fc2e7400700 1 -- 172.21.15.16:0/1420138072 wait complete. 2024-05-27T06:55:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.214+0000 7fc2e7400700 1 Processor -- start 2024-05-27T06:55:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.214+0000 7fc2e7400700 1 -- start start 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e01045e0 0x7fc2e0075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 0x7fc2e0076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0114390 con 0x7fc2e0108f10 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0114210 con 0x7fc2e01045e0 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2e7400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0114510 con 0x7fc2e0109860 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2dfe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2dfe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56528/0 (socket says 172.21.15.16:56528) 2024-05-27T06:55:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.215+0000 7fc2dfe00700 1 -- 172.21.15.16:0/4174241463 learned_addr learned my addr 172.21.15.16:0/4174241463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2e4a00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e01045e0 0x7fc2e0075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2e5400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 0x7fc2e0076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 -- 172.21.15.16:0/4174241463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 msgr2=0x7fc2e0076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 0x7fc2e0076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 -- 172.21.15.16:0/4174241463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e01045e0 msgr2=0x7fc2e0075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e01045e0 0x7fc2e0075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 -- 172.21.15.16:0/4174241463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e01a9140 con 0x7fc2e0108f10 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dfe00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00d980 tx=0x7fc2cc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.216+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc017070 con 0x7fc2e0108f10 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.217+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e01a9430 con 0x7fc2e0108f10 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.217+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2cc010460 con 0x7fc2e0108f10 2024-05-27T06:55:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.217+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc00de60 con 0x7fc2e0108f10 2024-05-27T06:55:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.217+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01a9970 con 0x7fc2e0108f10 2024-05-27T06:55:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.218+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e00512b0 con 0x7fc2e0108f10 2024-05-27T06:55:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.222+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc2cc010610 con 0x7fc2e0108f10 2024-05-27T06:55:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.224+0000 7fc2dd600700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 0x7fc2c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.224+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc2cc0e1c80 con 0x7fc2e0108f10 2024-05-27T06:55:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.225+0000 7fc2e4a00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 0x7fc2c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.225+0000 7fc2e4a00700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 0x7fc2c80b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000c00 tx=0x7fc2d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:18.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.227+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2cc0a9a50 con 0x7fc2e0108f10 2024-05-27T06:55:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:18.486+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fc2e002e5e0 con 0x7fc2e0108f10 2024-05-27T06:55:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.417+0000 7fc2dd600700 1 -- 172.21.15.16:0/4174241463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7fc2cc0a9350 con 0x7fc2e0108f10 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.421+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 msgr2=0x7fc2c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.421+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 0x7fc2c80b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000c00 tx=0x7fc2d4005fb0 comp rx=0 tx=0).stop 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.421+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 msgr2=0x7fc2e0075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.421+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00d980 tx=0x7fc2cc00dd40 comp rx=0 tx=0).stop 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 shutdown_connections 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc2c80b2880 0x7fc2c80b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e0109860 0x7fc2e0076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2e01045e0 0x7fc2e0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 --2- 172.21.15.16:0/4174241463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e0108f10 0x7fc2e0075ff0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 >> 172.21.15.16:0/4174241463 conn(0x7fc2e0100030 msgr2=0x7fc2e0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 shutdown_connections 2024-05-27T06:55:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:19.422+0000 7fc2e7400700 1 -- 172.21.15.16:0/4174241463 wait complete. 2024-05-27T06:55:19.623 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:55:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T06:55:19.274+0000 7f0ac5400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T06:55:19.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:19 smithi069 ceph-mon[35276]: pgmap v389: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 922 B/s wr, 5 op/s 2024-05-27T06:55:19.623 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:19 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T06:55:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:19 smithi164 ceph-mon[35327]: pgmap v389: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 922 B/s wr, 5 op/s 2024-05-27T06:55:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:19 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T06:55:19.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:19 smithi016 ceph-mon[31415]: pgmap v389: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 922 B/s wr, 5 op/s 2024-05-27T06:55:19.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:19 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T06:55:20.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T06:55:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:20 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T06:55:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:20 smithi016 ceph-mon[31415]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T06:55:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:20 smithi016 ceph-mon[31415]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:20.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:20 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T06:55:20.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:20 smithi164 ceph-mon[35327]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T06:55:20.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:20 smithi164 ceph-mon[35327]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:20.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:20 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4174241463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T06:55:20.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:20 smithi069 ceph-mon[35276]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T06:55:20.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:20 smithi069 ceph-mon[35276]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:23.279 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:23 smithi016 ceph-mon[31415]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 -- 172.21.15.16:0/3036317941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 msgr2=0x7f3e4c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 --2- 172.21.15.16:0/3036317941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c109300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e380099c0 tx=0x7f3e38009cd0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 -- 172.21.15.16:0/3036317941 shutdown_connections 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 --2- 172.21.15.16:0/3036317941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c109840 0x7f3e4c1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 --2- 172.21.15.16:0/3036317941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c1045c0 0x7f3e4c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 --2- 172.21.15.16:0/3036317941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c109300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 -- 172.21.15.16:0/3036317941 >> 172.21.15.16:0/3036317941 conn(0x7f3e4c0ffe90 msgr2=0x7f3e4c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 -- 172.21.15.16:0/3036317941 shutdown_connections 2024-05-27T06:55:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.404+0000 7f3e53e00700 1 -- 172.21.15.16:0/3036317941 wait complete. 2024-05-27T06:55:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.405+0000 7f3e53e00700 1 Processor -- start 2024-05-27T06:55:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 -- start start 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c1045c0 0x7f3e4c076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 0x7f3e4c077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114300 con 0x7f3e4c108ef0 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114180 con 0x7f3e4c109840 2024-05-27T06:55:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e53e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114480 con 0x7f3e4c1045c0 2024-05-27T06:55:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e50a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e50a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58264/0 (socket says 172.21.15.16:58264) 2024-05-27T06:55:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.406+0000 7f3e50a00700 1 -- 172.21.15.16:0/1159236730 learned_addr learned my addr 172.21.15.16:0/1159236730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e51400700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c1045c0 0x7f3e4c076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e51e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 0x7f3e4c077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 -- 172.21.15.16:0/1159236730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c1045c0 msgr2=0x7f3e4c076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c1045c0 0x7f3e4c076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 -- 172.21.15.16:0/1159236730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 msgr2=0x7f3e4c077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 0x7f3e4c077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 -- 172.21.15.16:0/1159236730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1a9020 con 0x7f3e4c108ef0 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e51e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 0x7f3e4c077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.407+0000 7f3e50a00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800b580 tx=0x7f3e3800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.408+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38026070 con 0x7f3e4c108ef0 2024-05-27T06:55:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.408+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e38005030 con 0x7f3e4c108ef0 2024-05-27T06:55:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.408+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c1a92b0 con 0x7f3e4c108ef0 2024-05-27T06:55:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.408+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380049f0 con 0x7f3e4c108ef0 2024-05-27T06:55:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.408+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1a9700 con 0x7f3e4c108ef0 2024-05-27T06:55:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.409+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c0512b0 con 0x7f3e4c108ef0 2024-05-27T06:55:23.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.412+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3e38029070 con 0x7f3e4c108ef0 2024-05-27T06:55:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.413+0000 7f3e4a000700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 0x7f3e340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.414+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f3e380e33d0 con 0x7f3e4c108ef0 2024-05-27T06:55:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.414+0000 7f3e51400700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 0x7f3e340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.415+0000 7f3e51400700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 0x7f3e340b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005ea0 tx=0x7f3e40005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.416+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e38013070 con 0x7f3e4c108ef0 2024-05-27T06:55:23.536 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:23 smithi164 ceph-mon[35327]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:23 smithi069 ceph-mon[35276]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T06:55:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:23.660+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3e4c02e850 con 0x7f3e4c108ef0 2024-05-27T06:55:24.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:24 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T06:55:24.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:24 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T06:55:24.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:24 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T06:55:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.185+0000 7f3e4a000700 1 -- 172.21.15.16:0/1159236730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f3e380ab0f0 con 0x7f3e4c108ef0 2024-05-27T06:55:25.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.189+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 msgr2=0x7f3e340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:25.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.189+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 0x7f3e340b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005ea0 tx=0x7f3e40005e30 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 msgr2=0x7f3e4c077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800b580 tx=0x7f3e3800f510 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 shutdown_connections 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3e340b2880 0x7f3e340b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4c109840 0x7f3e4c077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4c1045c0 0x7f3e4c076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 --2- 172.21.15.16:0/1159236730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c108ef0 0x7f3e4c077100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.190+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 >> 172.21.15.16:0/1159236730 conn(0x7f3e4c0ffe90 msgr2=0x7f3e4c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.191+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 shutdown_connections 2024-05-27T06:55:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:25.191+0000 7f3e53e00700 1 -- 172.21.15.16:0/1159236730 wait complete. 2024-05-27T06:55:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-05-27T06:55:25.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:25 smithi016 ceph-mon[31415]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T06:55:25.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:25 smithi016 ceph-mon[31415]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T06:55:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:25 smithi164 ceph-mon[35327]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T06:55:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:25 smithi164 ceph-mon[35327]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T06:55:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:25 smithi069 ceph-mon[35276]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T06:55:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:25 smithi069 ceph-mon[35276]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T06:55:26.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-27T06:55:26.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:26 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T06:55:26.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:26 smithi016 ceph-mon[31415]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T06:55:26.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:26 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T06:55:26.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:26 smithi164 ceph-mon[35327]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T06:55:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:26.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:26 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1159236730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T06:55:26.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:26 smithi069 ceph-mon[35276]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T06:55:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:27 smithi016 ceph-mon[31415]: pgmap v396: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:27 smithi016 ceph-mon[31415]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:27 smithi016 ceph-mon[31415]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T06:55:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:27 smithi016 ceph-mon[31415]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T06:55:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:27 smithi164 ceph-mon[35327]: pgmap v396: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:27 smithi164 ceph-mon[35327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:27 smithi164 ceph-mon[35327]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T06:55:27.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:27 smithi164 ceph-mon[35327]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T06:55:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:27 smithi069 ceph-mon[35276]: pgmap v396: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:27 smithi069 ceph-mon[35276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T06:55:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:27 smithi069 ceph-mon[35276]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T06:55:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:27 smithi069 ceph-mon[35276]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T06:55:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 -- 172.21.15.16:0/469868663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410b100 msgr2=0x7f8484117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 --2- 172.21.15.16:0/469868663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410b100 0x7f8484117610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f847c00b0a0 tx=0x7f847c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 -- 172.21.15.16:0/469868663 shutdown_connections 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 --2- 172.21.15.16:0/469868663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410b100 0x7f8484117610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 --2- 172.21.15.16:0/469868663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410f6b0 0x7f848410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 --2- 172.21.15.16:0/469868663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410a7b0 0x7f848410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 -- 172.21.15.16:0/469868663 >> 172.21.15.16:0/469868663 conn(0x7f8484076af0 msgr2=0x7f8484076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 -- 172.21.15.16:0/469868663 shutdown_connections 2024-05-27T06:55:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.971+0000 7f848b000700 1 -- 172.21.15.16:0/469868663 wait complete. 2024-05-27T06:55:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.972+0000 7f848b000700 1 Processor -- start 2024-05-27T06:55:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 -- start start 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 0x7f84841a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410b100 0x7f84841a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f848411a790 con 0x7f848410f6b0 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f848411a610 con 0x7f848410a7b0 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848b000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f848411a910 con 0x7f848410b100 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848a600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848a600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58290/0 (socket says 172.21.15.16:58290) 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.973+0000 7f848a600700 1 -- 172.21.15.16:0/2542203915 learned_addr learned my addr 172.21.15.16:0/2542203915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f8489c00700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 0x7f84841a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f8489200700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410b100 0x7f84841a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 -- 172.21.15.16:0/2542203915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410b100 msgr2=0x7f84841a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410b100 0x7f84841a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 -- 172.21.15.16:0/2542203915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 msgr2=0x7f84841a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 0x7f84841a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 -- 172.21.15.16:0/2542203915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84841a5f90 con 0x7f848410f6b0 2024-05-27T06:55:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f8489c00700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 0x7f84841a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.974+0000 7f848a600700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f847c00b070 tx=0x7f847c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.975+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c021070 con 0x7f848410f6b0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.975+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84841a6220 con 0x7f848410f6b0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.975+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f847c007c00 con 0x7f848410f6b0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.975+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c004850 con 0x7f848410f6b0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.975+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841adba0 con 0x7f848410f6b0 2024-05-27T06:55:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.976+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84840512b0 con 0x7f848410f6b0 2024-05-27T06:55:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.979+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f847c003fb0 con 0x7f848410f6b0 2024-05-27T06:55:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.980+0000 7f8482000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 0x7f846c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.980+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f847c00f020 con 0x7f848410f6b0 2024-05-27T06:55:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.981+0000 7f8489c00700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 0x7f846c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:28.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.982+0000 7f8489c00700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 0x7f846c0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8478009c90 tx=0x7f8478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:28.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:28.984+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f847c02a020 con 0x7f848410f6b0 2024-05-27T06:55:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:29.227+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f84841a2420 con 0x7f848410f6b0 2024-05-27T06:55:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:29 smithi164 ceph-mon[35327]: pgmap v399: 129 pgs: 4 creating+peering, 10 unknown, 115 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:29 smithi016 ceph-mon[31415]: pgmap v399: 129 pgs: 4 creating+peering, 10 unknown, 115 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:29 smithi069 ceph-mon[35276]: pgmap v399: 129 pgs: 4 creating+peering, 10 unknown, 115 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.233+0000 7f8482000700 1 -- 172.21.15.16:0/2542203915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f847c0ab8e0 con 0x7f848410f6b0 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 msgr2=0x7f846c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 0x7f846c0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8478009c90 tx=0x7f8478009260 comp rx=0 tx=0).stop 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 msgr2=0x7f84841a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f847c00b070 tx=0x7f847c004d30 comp rx=0 tx=0).stop 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 shutdown_connections 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f846c0b2880 0x7f846c0b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f848410f6b0 0x7f84841a58f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:30.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f848410a7b0 0x7f84841a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:30.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 --2- 172.21.15.16:0/2542203915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f848410b100 0x7f84841a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:30.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.239+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 >> 172.21.15.16:0/2542203915 conn(0x7f8484076af0 msgr2=0x7f8484108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:30.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.240+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 shutdown_connections 2024-05-27T06:55:30.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:30.241+0000 7f848b000700 1 -- 172.21.15.16:0/2542203915 wait complete. 2024-05-27T06:55:30.235 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-05-27T06:55:30.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:30 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T06:55:30.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:30 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T06:55:30.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:30 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T06:55:31.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-27T06:55:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:31 smithi164 ceph-mon[35327]: pgmap v400: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:31 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T06:55:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:31 smithi164 ceph-mon[35327]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T06:55:31.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:31 smithi016 ceph-mon[31415]: pgmap v400: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:31.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:31 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T06:55:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:31 smithi016 ceph-mon[31415]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T06:55:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:31 smithi069 ceph-mon[35276]: pgmap v400: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:31 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2542203915' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T06:55:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:31 smithi069 ceph-mon[35276]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T06:55:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:33 smithi164 ceph-mon[35327]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:33 smithi016 ceph-mon[31415]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:33.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:33 smithi069 ceph-mon[35276]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 -- 172.21.15.16:0/3853791492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109670 msgr2=0x7fe6c4110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 --2- 172.21.15.16:0/3853791492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109670 0x7fe6c4110fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00b0d0 tx=0x7fe6bc00b3e0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 -- 172.21.15.16:0/3853791492 shutdown_connections 2024-05-27T06:55:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 --2- 172.21.15.16:0/3853791492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109670 0x7fe6c4110fd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 --2- 172.21.15.16:0/3853791492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c41043f0 0x7fe6c4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 --2- 172.21.15.16:0/3853791492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c4108d20 0x7fe6c4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 -- 172.21.15.16:0/3853791492 >> 172.21.15.16:0/3853791492 conn(0x7fe6c4079d00 msgr2=0x7fe6c407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 -- 172.21.15.16:0/3853791492 shutdown_connections 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.251+0000 7fe6ca800700 1 -- 172.21.15.16:0/3853791492 wait complete. 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.252+0000 7fe6ca800700 1 Processor -- start 2024-05-27T06:55:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 -- start start 2024-05-27T06:55:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c41043f0 0x7fe6c41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 0x7fe6c419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41141a0 con 0x7fe6c4108d20 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4114020 con 0x7fe6c4109670 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6ca800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4114320 con 0x7fe6c41043f0 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.253+0000 7fe6c8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52564/0 (socket says 172.21.15.16:52564) 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 -- 172.21.15.16:0/873302798 learned_addr learned my addr 172.21.15.16:0/873302798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c9400700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c41043f0 0x7fe6c41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c9e00700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 0x7fe6c419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 -- 172.21.15.16:0/873302798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c41043f0 msgr2=0x7fe6c41a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c41043f0 0x7fe6c41a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 -- 172.21.15.16:0/873302798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 msgr2=0x7fe6c419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 0x7fe6c419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c8a00700 1 -- 172.21.15.16:0/873302798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c419b020 con 0x7fe6c4108d20 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.254+0000 7fe6c9e00700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 0x7fe6c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6c8a00700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c9e0 tx=0x7fe6b000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0007aa0 con 0x7fe6c4108d20 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6b0007c40 con 0x7fe6c4108d20 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018660 con 0x7fe6c4108d20 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c419b310 con 0x7fe6c4108d20 2024-05-27T06:55:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.255+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c419b800 con 0x7fe6c4108d20 2024-05-27T06:55:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.256+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c41058a0 con 0x7fe6c4108d20 2024-05-27T06:55:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.259+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe6b000f450 con 0x7fe6c4108d20 2024-05-27T06:55:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.260+0000 7fe6c2000700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 0x7fe6ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.260+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7fe6b00e2670 con 0x7fe6c4108d20 2024-05-27T06:55:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.261+0000 7fe6c9400700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 0x7fe6ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:34.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.261+0000 7fe6c9400700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 0x7fe6ac0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4102c30 tx=0x7fe6b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.263+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6b0014030 con 0x7fe6c4108d20 2024-05-27T06:55:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:34.550+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe6c419bec0 con 0x7fe6c4108d20 2024-05-27T06:55:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.251+0000 7fe6c2000700 1 -- 172.21.15.16:0/873302798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fe6b00aa1a0 con 0x7fe6c4108d20 2024-05-27T06:55:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.256+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 msgr2=0x7fe6ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.256+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 0x7fe6ac0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4102c30 tx=0x7fe6b800b410 comp rx=0 tx=0).stop 2024-05-27T06:55:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.256+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 msgr2=0x7fe6c41a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.256+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c9e0 tx=0x7fe6b000ccf0 comp rx=0 tx=0).stop 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 shutdown_connections 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe6ac0b2880 0x7fe6ac0b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6c4109670 0x7fe6c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c41043f0 0x7fe6c41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 --2- 172.21.15.16:0/873302798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108d20 0x7fe6c41a0cb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.257+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 >> 172.21.15.16:0/873302798 conn(0x7fe6c4079d00 msgr2=0x7fe6c4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.258+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 shutdown_connections 2024-05-27T06:55:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:35.258+0000 7fe6ca800700 1 -- 172.21.15.16:0/873302798 wait complete. 2024-05-27T06:55:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-05-27T06:55:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:35 smithi164 ceph-mon[35327]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:35 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T06:55:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:35 smithi016 ceph-mon[31415]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:35 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T06:55:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:35 smithi069 ceph-mon[35276]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:35 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T06:55:36.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:55:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:36 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T06:55:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:36 smithi016 ceph-mon[31415]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T06:55:36.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:36 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:36 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T06:55:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:36 smithi164 ceph-mon[35327]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T06:55:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:36 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:36.584 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:36 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/873302798' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T06:55:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:36 smithi069 ceph-mon[35276]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T06:55:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:36 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:55:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:37 smithi016 ceph-mon[31415]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:37 smithi016 ceph-mon[31415]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:55:37.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:37 smithi016 ceph-mon[31415]: Cluster is now healthy 2024-05-27T06:55:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:37 smithi164 ceph-mon[35327]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:37 smithi164 ceph-mon[35327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:55:37.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:37 smithi164 ceph-mon[35327]: Cluster is now healthy 2024-05-27T06:55:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:37 smithi069 ceph-mon[35276]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:37 smithi069 ceph-mon[35276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T06:55:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:37 smithi069 ceph-mon[35276]: Cluster is now healthy 2024-05-27T06:55:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T06:55:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T06:55:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T06:55:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T06:55:38.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-27T06:55:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-27T06:55:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-27T06:55:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-27T06:55:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-27T06:55:38.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:38 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T06:55:38.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-27T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:38 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T06:55:38.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T06:55:38.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T06:55:38.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T06:55:38.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-27T06:55:38.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:38 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 -- 172.21.15.16:0/2779966980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab410b2d0 msgr2=0x7ffab4117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2779966980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab410b2d0 0x7ffab4117800 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b0a0 tx=0x7ffaac00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 -- 172.21.15.16:0/2779966980 shutdown_connections 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2779966980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab410b2d0 0x7ffab4117800 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2779966980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab4110960 0x7ffab4110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2779966980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 0x7ffab410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.962+0000 7ffab8e00700 1 -- 172.21.15.16:0/2779966980 >> 172.21.15.16:0/2779966980 conn(0x7ffab4077c70 msgr2=0x7ffab40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.963+0000 7ffab8e00700 1 -- 172.21.15.16:0/2779966980 shutdown_connections 2024-05-27T06:55:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.963+0000 7ffab8e00700 1 -- 172.21.15.16:0/2779966980 wait complete. 2024-05-27T06:55:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.964+0000 7ffab8e00700 1 Processor -- start 2024-05-27T06:55:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.964+0000 7ffab8e00700 1 -- start start 2024-05-27T06:55:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 0x7ffab41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 0x7ffab41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffab411a930 con 0x7ffab4110960 2024-05-27T06:55:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffab411a7b0 con 0x7ffab410a980 2024-05-27T06:55:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab8e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffab411aab0 con 0x7ffab410b2d0 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52576/0 (socket says 172.21.15.16:52576) 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab2800700 1 -- 172.21.15.16:0/2845880208 learned_addr learned my addr 172.21.15.16:0/2845880208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.965+0000 7ffab1e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 0x7ffab41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab1400700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 0x7ffab41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 -- 172.21.15.16:0/2845880208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 msgr2=0x7ffab41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 0x7ffab41a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 -- 172.21.15.16:0/2845880208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 msgr2=0x7ffab41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 0x7ffab41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 -- 172.21.15.16:0/2845880208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41a61f0 con 0x7ffab4110960 2024-05-27T06:55:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab1400700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 0x7ffab41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.966+0000 7ffab2800700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0062a0 tx=0x7ffaac004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.967+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac021070 con 0x7ffab4110960 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.967+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaac007c80 con 0x7ffab4110960 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.967+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0052a0 con 0x7ffab4110960 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.967+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a6480 con 0x7ffab4110960 2024-05-27T06:55:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.967+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a6910 con 0x7ffab4110960 2024-05-27T06:55:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.968+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab40512b0 con 0x7ffab4110960 2024-05-27T06:55:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.971+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffaac024070 con 0x7ffab4110960 2024-05-27T06:55:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.973+0000 7ffaaaa00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 0x7ffa980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.973+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ffaac00f020 con 0x7ffab4110960 2024-05-27T06:55:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.973+0000 7ffab1e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 0x7ffa980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.974+0000 7ffab1e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 0x7ffa980b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005950 tx=0x7ffaa400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:38.976+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffaac02b020 con 0x7ffab4110960 2024-05-27T06:55:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.265+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffab402e5e0 con 0x7ffab4110960 2024-05-27T06:55:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.267+0000 7ffaaaa00700 1 -- 172.21.15.16:0/2845880208 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7ffaac0aba80 con 0x7ffab4110960 2024-05-27T06:55:39.261 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:55:39.261 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:55:34.552848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34517,"qdb_cluster":[34517]},"id":1}]} 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 msgr2=0x7ffa980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 0x7ffa980b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005950 tx=0x7ffaa400a300 comp rx=0 tx=0).stop 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 msgr2=0x7ffab41a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0062a0 tx=0x7ffaac004e20 comp rx=0 tx=0).stop 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 shutdown_connections 2024-05-27T06:55:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ffa980b2900 0x7ffa980b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffab4110960 0x7ffab41a5b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab410a980 0x7ffab41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 --2- 172.21.15.16:0/2845880208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab410b2d0 0x7ffab41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.271+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 >> 172.21.15.16:0/2845880208 conn(0x7ffab4077c70 msgr2=0x7ffab407b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.272+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 shutdown_connections 2024-05-27T06:55:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:39.272+0000 7ffab8e00700 1 -- 172.21.15.16:0/2845880208 wait complete. 2024-05-27T06:55:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-27T06:55:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-27T06:55:39.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T06:55:39.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T06:55:39.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:39 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2845880208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T06:55:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T06:55:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T06:55:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:39 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2845880208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T06:55:39.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:39 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2845880208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:40.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:55:40.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:40 smithi016 ceph-mon[31415]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T06:55:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:40 smithi164 ceph-mon[35327]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T06:55:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:40 smithi069 ceph-mon[35276]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T06:55:40.633 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:41 smithi164 ceph-mon[35327]: pgmap v409: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:41.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:41 smithi016 ceph-mon[31415]: pgmap v409: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:41.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:41 smithi069 ceph-mon[35276]: pgmap v409: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 -- 172.21.15.16:0/3254395526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc194106eb0 msgr2=0x7fc194113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 --2- 172.21.15.16:0/3254395526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc194106eb0 0x7fc194113340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b0a0 tx=0x7fc18c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 -- 172.21.15.16:0/3254395526 shutdown_connections 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 --2- 172.21.15.16:0/3254395526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc194106eb0 0x7fc194113340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 --2- 172.21.15.16:0/3254395526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc19410c5a0 0x7fc19410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 --2- 172.21.15.16:0/3254395526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106560 0x7fc194106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 -- 172.21.15.16:0/3254395526 >> 172.21.15.16:0/3254395526 conn(0x7fc194102040 msgr2=0x7fc194104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 -- 172.21.15.16:0/3254395526 shutdown_connections 2024-05-27T06:55:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.040+0000 7fc19b200700 1 -- 172.21.15.16:0/3254395526 wait complete. 2024-05-27T06:55:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.042+0000 7fc19b200700 1 Processor -- start 2024-05-27T06:55:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.042+0000 7fc19b200700 1 -- start start 2024-05-27T06:55:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194106560 0x7fc19419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106eb0 0x7fc19419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941165a0 con 0x7fc19410c5a0 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc194116420 con 0x7fc194106560 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.043+0000 7fc19b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc194116720 con 0x7fc194106eb0 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35366/0 (socket says 172.21.15.16:35366) 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 -- 172.21.15.16:0/1372724812 learned_addr learned my addr 172.21.15.16:0/1372724812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc193e00700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194106560 0x7fc19419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc18be00700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106eb0 0x7fc19419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 -- 172.21.15.16:0/1372724812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106eb0 msgr2=0x7fc19419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106eb0 0x7fc19419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 -- 172.21.15.16:0/1372724812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194106560 msgr2=0x7fc19419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194106560 0x7fc19419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.044+0000 7fc199200700 1 -- 172.21.15.16:0/1372724812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941a1d60 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc199200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b830 tx=0x7fc18c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c021070 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941a1ff0 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18c007b80 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0052a0 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.045+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a24d0 con 0x7fc19410c5a0 2024-05-27T06:55:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.047+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19410e960 con 0x7fc19410c5a0 2024-05-27T06:55:43.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.049+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc18c024070 con 0x7fc19410c5a0 2024-05-27T06:55:43.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.051+0000 7fc191600700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 0x7fc1780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:43.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.051+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc18c00f020 con 0x7fc19410c5a0 2024-05-27T06:55:43.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.051+0000 7fc193e00700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 0x7fc1780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:43.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.052+0000 7fc193e00700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 0x7fc1780b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc184005950 tx=0x7fc18400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.056+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc18c02b020 con 0x7fc19410c5a0 2024-05-27T06:55:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.296+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19419e1f0 con 0x7fc19410c5a0 2024-05-27T06:55:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.298+0000 7fc191600700 1 -- 172.21.15.16:0/1372724812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24566 (secure 0 0 0) 0x7fc18c0ab540 con 0x7fc19410c5a0 2024-05-27T06:55:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:55:43.292 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:55:39.287742+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 msgr2=0x7fc1780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 0x7fc1780b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc184005950 tx=0x7fc18400b410 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 msgr2=0x7fc1941a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b830 tx=0x7fc18c004d00 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 shutdown_connections 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc1780b2880 0x7fc1780b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19410c5a0 0x7fc1941a16c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194106560 0x7fc19419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 --2- 172.21.15.16:0/1372724812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc194106eb0 0x7fc19419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 >> 172.21.15.16:0/1372724812 conn(0x7fc194102040 msgr2=0x7fc1941037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 shutdown_connections 2024-05-27T06:55:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:43.302+0000 7fc19b200700 1 -- 172.21.15.16:0/1372724812 wait complete. 2024-05-27T06:55:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:43 smithi164 ceph-mon[35327]: pgmap v410: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T06:55:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1372724812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:43 smithi016 ceph-mon[31415]: pgmap v410: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T06:55:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1372724812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:43 smithi069 ceph-mon[35276]: pgmap v410: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T06:55:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1372724812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:44.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:55:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:45 smithi164 ceph-mon[35327]: pgmap v411: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:45 smithi016 ceph-mon[31415]: pgmap v411: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:45 smithi069 ceph-mon[35276]: pgmap v411: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.114+0000 7f1769800700 1 -- 172.21.15.16:0/1059721078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106ec0 msgr2=0x7f1764113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.114+0000 7f1769800700 1 --2- 172.21.15.16:0/1059721078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106ec0 0x7f1764113350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b0a0 tx=0x7f175c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 -- 172.21.15.16:0/1059721078 shutdown_connections 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 --2- 172.21.15.16:0/1059721078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106ec0 0x7f1764113350 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 --2- 172.21.15.16:0/1059721078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 0x7f176410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 --2- 172.21.15.16:0/1059721078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106570 0x7f1764106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 -- 172.21.15.16:0/1059721078 >> 172.21.15.16:0/1059721078 conn(0x7f1764102070 msgr2=0x7f17641044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 -- 172.21.15.16:0/1059721078 shutdown_connections 2024-05-27T06:55:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.115+0000 7f1769800700 1 -- 172.21.15.16:0/1059721078 wait complete. 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.116+0000 7f1769800700 1 Processor -- start 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.116+0000 7f1769800700 1 -- start start 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.116+0000 7f1769800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.116+0000 7f1769800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106ec0 0x7f176419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1769800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 0x7f17641a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1769800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17641165e0 con 0x7f1764106570 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1769800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1764116460 con 0x7f1764106ec0 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1769800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1764116760 con 0x7f176410c5b0 2024-05-27T06:55:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35380/0 (socket says 172.21.15.16:35380) 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 -- 172.21.15.16:0/2818018177 learned_addr learned my addr 172.21.15.16:0/2818018177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762000700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106ec0 0x7f176419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1763400700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 0x7f17641a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 -- 172.21.15.16:0/2818018177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 msgr2=0x7f17641a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 0x7f17641a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 -- 172.21.15.16:0/2818018177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106ec0 msgr2=0x7f176419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106ec0 0x7f176419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.117+0000 7f1762a00700 1 -- 172.21.15.16:0/2818018177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17641a1de0 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f1762a00700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f175400e970 tx=0x7f175400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175400cbd0 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f175400cd70 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17641a20d0 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754005610 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.118+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17641a99f0 con 0x7f1764106570 2024-05-27T06:55:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.119+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17640512b0 con 0x7f1764106570 2024-05-27T06:55:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.122+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1754004750 con 0x7f1764106570 2024-05-27T06:55:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.124+0000 7f175b400700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 0x7f17480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.124+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f17540e2b40 con 0x7f1764106570 2024-05-27T06:55:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.124+0000 7f1762000700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 0x7f17480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.125+0000 7f1762000700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 0x7f17480b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1764103da0 tx=0x7f1750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.127+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1754054df0 con 0x7f1764106570 2024-05-27T06:55:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.367+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f176419e270 con 0x7f1764106570 2024-05-27T06:55:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.370+0000 7f175b400700 1 -- 172.21.15.16:0/2818018177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24566 (secure 0 0 0) 0x7f17540aa4a0 con 0x7f1764106570 2024-05-27T06:55:47.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:55:47.364 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:55:39.287742+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:55:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 msgr2=0x7f17480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 0x7f17480b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1764103da0 tx=0x7f1750009260 comp rx=0 tx=0).stop 2024-05-27T06:55:47.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 msgr2=0x7f176419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:47.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f175400e970 tx=0x7f175400ed30 comp rx=0 tx=0).stop 2024-05-27T06:55:47.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:47 smithi016 ceph-mon[31415]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 shutdown_connections 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f17480b2880 0x7f17480b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176410c5b0 0x7f17641a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1764106570 0x7f176419cf10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 --2- 172.21.15.16:0/2818018177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764106ec0 0x7f176419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.374+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 >> 172.21.15.16:0/2818018177 conn(0x7f1764102070 msgr2=0x7f1764104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.375+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 shutdown_connections 2024-05-27T06:55:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:47.375+0000 7f1769800700 1 -- 172.21.15.16:0/2818018177 wait complete. 2024-05-27T06:55:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:47 smithi069 ceph-mon[35276]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:47 smithi164 ceph-mon[35327]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-05-27T06:55:48.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:55:48.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:48 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2818018177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:48.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:48 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2818018177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:48.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:48 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2818018177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:49 smithi016 ceph-mon[31415]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-05-27T06:55:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:49 smithi069 ceph-mon[35276]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-05-27T06:55:49.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:49 smithi164 ceph-mon[35327]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-05-27T06:55:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 -- 172.21.15.16:0/3085468913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 msgr2=0x7f83f0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 --2- 172.21.15.16:0/3085468913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f0106980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009990 tx=0x7f83dc009ca0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 -- 172.21.15.16:0/3085468913 shutdown_connections 2024-05-27T06:55:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 --2- 172.21.15.16:0/3085468913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f0106ec0 0x7f83f0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 --2- 172.21.15.16:0/3085468913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f010c5b0 0x7f83f010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 --2- 172.21.15.16:0/3085468913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f0106980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 -- 172.21.15.16:0/3085468913 >> 172.21.15.16:0/3085468913 conn(0x7f83f0102070 msgr2=0x7f83f01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 -- 172.21.15.16:0/3085468913 shutdown_connections 2024-05-27T06:55:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.064+0000 7f83f7800700 1 -- 172.21.15.16:0/3085468913 wait complete. 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.065+0000 7f83f7800700 1 Processor -- start 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.065+0000 7f83f7800700 1 -- start start 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f0106ec0 0x7f83f019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f010c5b0 0x7f83f01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0116590 con 0x7f83f0106570 2024-05-27T06:55:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0116410 con 0x7f83f010c5b0 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f7800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0116710 con 0x7f83f0106ec0 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35400/0 (socket says 172.21.15.16:35400) 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.066+0000 7f83f4e00700 1 -- 172.21.15.16:0/3953274079 learned_addr learned my addr 172.21.15.16:0/3953274079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83efe00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f0106ec0 0x7f83f019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f5800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f010c5b0 0x7f83f01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 -- 172.21.15.16:0/3953274079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f0106ec0 msgr2=0x7f83f019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f0106ec0 0x7f83f019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 -- 172.21.15.16:0/3953274079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f010c5b0 msgr2=0x7f83f01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f010c5b0 0x7f83f01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 -- 172.21.15.16:0/3953274079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f01a1f10 con 0x7f83f0106570 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.067+0000 7f83f4e00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d980 tx=0x7f83e400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.068+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4017070 con 0x7f83f0106570 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.068+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83e4010460 con 0x7f83f0106570 2024-05-27T06:55:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.068+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f01a2200 con 0x7f83f0106570 2024-05-27T06:55:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.068+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e400de60 con 0x7f83f0106570 2024-05-27T06:55:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.068+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f01a9b20 con 0x7f83f0106570 2024-05-27T06:55:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.069+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f00512b0 con 0x7f83f0106570 2024-05-27T06:55:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.071+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f83e4010610 con 0x7f83f0106570 2024-05-27T06:55:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.073+0000 7f83ed600700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 0x7f83d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.073+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f83e40e1fe0 con 0x7f83f0106570 2024-05-27T06:55:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.074+0000 7f83efe00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 0x7f83d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.074+0000 7f83efe00700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 0x7f83d80b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103ed0 tx=0x7f83dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.076+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83e40a99c0 con 0x7f83f0106570 2024-05-27T06:55:51.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:51 smithi016 ceph-mon[31415]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-05-27T06:55:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.366+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83f019e3d0 con 0x7f83f0106570 2024-05-27T06:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.374+0000 7f83ed600700 1 -- 172.21.15.16:0/3953274079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7f83f019e3d0 con 0x7f83f0106570 2024-05-27T06:55:51.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:55:51.367 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:55:34.552848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34517,"qdb_cluster":[34517]},"id":1}]} 2024-05-27T06:55:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 msgr2=0x7f83d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 0x7f83d80b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103ed0 tx=0x7f83dc005fb0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 msgr2=0x7f83f019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d980 tx=0x7f83e400dd40 comp rx=0 tx=0).stop 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 shutdown_connections 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f83d80b2900 0x7f83d80b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f010c5b0 0x7f83f01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f0106570 0x7f83f019d040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 --2- 172.21.15.16:0/3953274079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f0106ec0 0x7f83f019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.377+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 >> 172.21.15.16:0/3953274079 conn(0x7f83f0102070 msgr2=0x7f83f01049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.378+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 shutdown_connections 2024-05-27T06:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:51.378+0000 7f83f7800700 1 -- 172.21.15.16:0/3953274079 wait complete. 2024-05-27T06:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-05-27T06:55:51.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:51 smithi069 ceph-mon[35276]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-05-27T06:55:51.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:51 smithi164 ceph-mon[35327]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-05-27T06:55:52.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:55:52.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:52 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3953274079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:52.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:52 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3953274079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:52.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:52 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3953274079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:55:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:53 smithi016 ceph-mon[31415]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:53.734 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:53 smithi164 ceph-mon[35327]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:53.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:53 smithi069 ceph-mon[35276]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 -- 172.21.15.16:0/2599295956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 msgr2=0x7f0c88110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 --2- 172.21.15.16:0/2599295956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c88110d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800b0a0 tx=0x7f0c7800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 -- 172.21.15.16:0/2599295956 shutdown_connections 2024-05-27T06:55:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 --2- 172.21.15.16:0/2599295956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810b2d0 0x7f0c88117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 --2- 172.21.15.16:0/2599295956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c88110d70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 --2- 172.21.15.16:0/2599295956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810a980 0x7f0c8810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 -- 172.21.15.16:0/2599295956 >> 172.21.15.16:0/2599295956 conn(0x7f0c88077c70 msgr2=0x7f0c880780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 -- 172.21.15.16:0/2599295956 shutdown_connections 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.051+0000 7f0c8e400700 1 -- 172.21.15.16:0/2599295956 wait complete. 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 Processor -- start 2024-05-27T06:55:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 -- start start 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810a980 0x7f0c881a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810b2d0 0x7f0c881a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8811a9e0 con 0x7f0c88110960 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8811a860 con 0x7f0c8810a980 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.053+0000 7f0c8e400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8811ab60 con 0x7f0c8810b2d0 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54424/0 (socket says 172.21.15.16:54424) 2024-05-27T06:55:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 -- 172.21.15.16:0/1675026465 learned_addr learned my addr 172.21.15.16:0/1675026465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810a980 0x7f0c881a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c7ea00700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810b2d0 0x7f0c881a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 -- 172.21.15.16:0/1675026465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810b2d0 msgr2=0x7f0c881a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810b2d0 0x7f0c881a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 -- 172.21.15.16:0/1675026465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810a980 msgr2=0x7f0c881a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810a980 0x7f0c881a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.054+0000 7f0c87e00700 1 -- 172.21.15.16:0/1675026465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c881a6230 con 0x7f0c88110960 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c87e00700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c900 tx=0x7f0c8000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800078d0 con 0x7f0c88110960 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c8000f450 con 0x7f0c88110960 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c881a64c0 con 0x7f0c88110960 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80018660 con 0x7f0c88110960 2024-05-27T06:55:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.055+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c881a6a00 con 0x7f0c88110960 2024-05-27T06:55:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.057+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c880512b0 con 0x7f0c88110960 2024-05-27T06:55:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.059+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0c80007a70 con 0x7f0c88110960 2024-05-27T06:55:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.061+0000 7f0c84c00700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 0x7f0c6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.061+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0c800e2460 con 0x7f0c88110960 2024-05-27T06:55:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.061+0000 7f0c87400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 0x7f0c6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.062+0000 7f0c87400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 0x7f0c6c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800b830 tx=0x7f0c78006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.064+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c80014030 con 0x7f0c88110960 2024-05-27T06:55:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.305+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c8802e5e0 con 0x7f0c88110960 2024-05-27T06:55:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.310+0000 7f0c84c00700 1 -- 172.21.15.16:0/1675026465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24566 (secure 0 0 0) 0x7f0c800a9ed0 con 0x7f0c88110960 2024-05-27T06:55:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:55:55.304 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:55:39.287742+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:55:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 msgr2=0x7f0c6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 0x7f0c6c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800b830 tx=0x7f0c78006210 comp rx=0 tx=0).stop 2024-05-27T06:55:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 msgr2=0x7f0c881a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c900 tx=0x7f0c8000cc10 comp rx=0 tx=0).stop 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 shutdown_connections 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0c6c0b2900 0x7f0c6c0b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c88110960 0x7f0c881a5c40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c8810a980 0x7f0c881a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 --2- 172.21.15.16:0/1675026465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8810b2d0 0x7f0c881a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 >> 172.21.15.16:0/1675026465 conn(0x7f0c88077c70 msgr2=0x7f0c8807bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 shutdown_connections 2024-05-27T06:55:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:55.314+0000 7f0c8e400700 1 -- 172.21.15.16:0/1675026465 wait complete. 2024-05-27T06:55:55.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:55 smithi016 ceph-mon[31415]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:55 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1675026465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:55.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:55 smithi164 ceph-mon[35327]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:55.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:55 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1675026465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:55.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:55 smithi069 ceph-mon[35276]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-05-27T06:55:55.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:55 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1675026465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:55:56.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T06:55:13.518131+0000', 'modified': '2024-05-27T06:55:34.552848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.69:6839/962503578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 962503578}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 962503578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34517, 'qdb_cluster': [34517]} 2024-05-27T06:55:56.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T06:55:56.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T06:55:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:55:57.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:57 smithi016 ceph-mon[31415]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:57.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:57 smithi164 ceph-mon[35327]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:57.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:57 smithi069 ceph-mon[35276]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 -- 172.21.15.16:0/2636131140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca010c5a0 msgr2=0x7f9ca010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 --2- 172.21.15.16:0/2636131140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca010c5a0 0x7f9ca010c9b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009990 tx=0x7f9c94009ca0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 -- 172.21.15.16:0/2636131140 shutdown_connections 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 --2- 172.21.15.16:0/2636131140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0106eb0 0x7f9ca0113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 --2- 172.21.15.16:0/2636131140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca010c5a0 0x7f9ca010c9b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 --2- 172.21.15.16:0/2636131140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106560 0x7f9ca0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 -- 172.21.15.16:0/2636131140 >> 172.21.15.16:0/2636131140 conn(0x7f9ca0102040 msgr2=0x7f9ca0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 -- 172.21.15.16:0/2636131140 shutdown_connections 2024-05-27T06:55:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.060+0000 7f9ca8400700 1 -- 172.21.15.16:0/2636131140 wait complete. 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.061+0000 7f9ca8400700 1 Processor -- start 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.061+0000 7f9ca8400700 1 -- start start 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106eb0 0x7f9ca019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 0x7f9ca01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01165b0 con 0x7f9ca0106560 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0116430 con 0x7f9ca010c5a0 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca8400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0116730 con 0x7f9ca0106eb0 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54442/0 (socket says 172.21.15.16:54442) 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.062+0000 7f9ca5a00700 1 -- 172.21.15.16:0/688904983 learned_addr learned my addr 172.21.15.16:0/688904983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5000700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106eb0 0x7f9ca019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca6400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 0x7f9ca01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 -- 172.21.15.16:0/688904983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106eb0 msgr2=0x7f9ca019d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106eb0 0x7f9ca019d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 -- 172.21.15.16:0/688904983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 msgr2=0x7f9ca01a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 0x7f9ca01a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 -- 172.21.15.16:0/688904983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01a1da0 con 0x7f9ca0106560 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca6400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 0x7f9ca01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:55:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.063+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c940036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.064+0000 7f9c9e000700 1 -- 172.21.15.16:0/688904983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94027070 con 0x7f9ca0106560 2024-05-27T06:55:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.064+0000 7f9c9e000700 1 -- 172.21.15.16:0/688904983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c9400fc80 con 0x7f9ca0106560 2024-05-27T06:55:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.064+0000 7f9c9e000700 1 -- 172.21.15.16:0/688904983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94022d30 con 0x7f9ca0106560 2024-05-27T06:55:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.064+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01a2030 con 0x7f9ca0106560 2024-05-27T06:55:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.064+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a2560 con 0x7f9ca0106560 2024-05-27T06:55:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.068+0000 7f9c9e000700 1 -- 172.21.15.16:0/688904983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9c9402a070 con 0x7f9ca0106560 2024-05-27T06:55:59.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 Processor -- start 2024-05-27T06:55:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 -- start start 2024-05-27T06:55:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9c9e000700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 0x7f9c880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562318209df0 con 0x56231815cdf0 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.069+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5623181f9990 con 0x5623182e1ce0 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623181fd9d0 con 0x5623182dbd60 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9c9e000700 1 -- 172.21.15.16:0/688904983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9c940e3500 con 0x7f9ca0106560 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca6400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 0x7f9c880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54458/0 (socket says 172.21.15.16:54458) 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca5a00700 1 -- 172.21.15.16:0/2016445038 learned_addr learned my addr 172.21.15.16:0/2016445038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca6400700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca5000700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.070+0000 7f9ca5a00700 1 -- 172.21.15.16:0/2016445038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 msgr2=0x5623182dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca5a00700 1 -- 172.21.15.16:0/2016445038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 msgr2=0x5623182db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182db820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca5a00700 1 -- 172.21.15.16:0/2016445038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623181c23d0 con 0x56231815cdf0 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca6400700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9c940aa780 tx=0x7f9c94022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3828fa1a33d49811 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9ca6400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 0x7f9c880b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c98012780 tx=0x7f9c98007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9c92a00700 1 -- 172.21.15.16:0/2016445038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940039a0 con 0x56231815cdf0 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9c92a00700 1 -- 172.21.15.16:0/2016445038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c940cd0b0 con 0x56231815cdf0 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.071+0000 7f9c92a00700 1 -- 172.21.15.16:0/2016445038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940037c0 con 0x56231815cdf0 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.072+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/2016445038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 msgr2=0x5623182e17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.072+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9c940aa780 tx=0x7f9c94022470 comp rx=0 tx=0).stop 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/2016445038 shutdown_connections 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182e17a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/2016445038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/2016445038 >> 172.21.15.16:0/2016445038 conn(0x5623181bcf70 msgr2=0x5623182d6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/2016445038 shutdown_connections 2024-05-27T06:55:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/2016445038 wait complete. 2024-05-27T06:55:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.073+0000 7f9cb6e81b80 1 -- start start 2024-05-27T06:55:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 Processor -- start 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 0x5623182e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623180fff50 con 0x5623182dbd60 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56231819a710 con 0x5623182e1ce0 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623182e8570 con 0x56231815cdf0 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.074+0000 7f9cb6e81b80 10 client.0 _ll_register_callbacks cb 0x5623181a3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54468/0 (socket says 172.21.15.16:54468) 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 -- 172.21.15.16:0/724408010 learned_addr learned my addr 172.21.15.16:0/724408010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 0x5623182e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca6400700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 -- 172.21.15.16:0/724408010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 msgr2=0x5623182e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 0x5623182e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 -- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 msgr2=0x5623182e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5000700 1 -- 172.21.15.16:0/724408010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562318149880 con 0x5623182dbd60 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca6400700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.075+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 0x5623182e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9ca5000700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c011810 tx=0x7f9c8c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cd575f9f127161 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9c87400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c01e060 con 0x5623182dbd60 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c940cd0b0 con 0x5623182dbd60 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9c8c00fad0 con 0x5623182dbd60 2024-05-27T06:55:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c025050 con 0x5623182dbd60 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9cb6e81b80 10 client.14703 fetch_fsmap learned FSMap version 17 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.076+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5623181ca010 con 0x5623182dbd60 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c940cd0b0 con 0x5623182dbd60 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f9c8c018030 con 0x5623182dbd60 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 10 client.14703 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 10 client.14703 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 20 client.14703 populate_metadata read hostname 'smithi016' 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5623182f5400 0x5623182f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 4 client.14703 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.077+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231820bd40 con 0x5623182f5400 2024-05-27T06:55:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.078+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5623182f5400 0x5623182f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.079+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5623182f5400 0x5623182f6870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac7bf8874c09414 server_cookie=699973800c779790 in_seq=0 out_seq=0 2024-05-27T06:55:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.079+0000 7f9c87400700 10 client.14703 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T06:55:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9c87400700 1 -- 172.21.15.16:0/724408010 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56231820bd40 con 0x5623182f5400 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9c87400700 10 client.14703 handle_command_reply: tid=0 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 client.14703 shutdown 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 msgr2=0x5623182e5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c011810 tx=0x7f9c8c011bd0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 shutdown_connections 2024-05-27T06:55:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5623182f5400 0x5623182f6870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56231815cdf0 0x5623182e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/724408010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623182dbd60 0x5623182e5940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9c87400700 0 client.14703 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 >> 172.21.15.16:0/724408010 conn(0x5623181bcf70 msgr2=0x5623182d5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 shutdown_connections 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.082+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/724408010 wait complete. 2024-05-27T06:55:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.083+0000 7f9cb6e81b80 20 client.14703 trim_cache size 0 max 16384 2024-05-27T06:55:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.106+0000 7f9cb6e81b80 1 Processor -- start 2024-05-27T06:55:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.106+0000 7f9cb6e81b80 1 -- start start 2024-05-27T06:55:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623182f4bb0 con 0x56231815cdf0 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5623182f5280 con 0x5623182e1ce0 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623182f5950 con 0x5623182dbd60 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54472/0 (socket says 172.21.15.16:54472) 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.107+0000 7f9ca5a00700 1 -- 172.21.15.16:0/108112125 learned_addr learned my addr 172.21.15.16:0/108112125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5000700 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca6400700 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 -- 172.21.15.16:0/108112125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 msgr2=0x5623182f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 -- 172.21.15.16:0/108112125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 msgr2=0x5623182e0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 -- 172.21.15.16:0/108112125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623182f2db0 con 0x56231815cdf0 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.108+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c940aa4f0 tx=0x7f9c940cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a6fb0d94becf0a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.109+0000 7f9c92000700 1 -- 172.21.15.16:0/108112125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940c3070 con 0x56231815cdf0 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.109+0000 7f9c92000700 1 -- 172.21.15.16:0/108112125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c940aaef0 con 0x56231815cdf0 2024-05-27T06:55:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.109+0000 7f9c92000700 1 -- 172.21.15.16:0/108112125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940e3040 con 0x56231815cdf0 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/108112125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 msgr2=0x5623182db820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c940aa4f0 tx=0x7f9c940cd040 comp rx=0 tx=0).stop 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/108112125 shutdown_connections 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182e1ce0 0x5623182e0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182db820 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/108112125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182dbd60 0x5623182f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/108112125 >> 172.21.15.16:0/108112125 conn(0x5623181bcf70 msgr2=0x5623182e0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/108112125 shutdown_connections 2024-05-27T06:55:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/108112125 wait complete. 2024-05-27T06:55:59.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.110+0000 7f9cb6e81b80 1 -- start start 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 Processor -- start 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182dbd60 0x5623182f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 0x5623182fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623182f66f0 con 0x56231815cdf0 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5623182f6020 con 0x5623182dbd60 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623182f6dc0 con 0x5623182e1ce0 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9cb6e81b80 10 client.0 _ll_register_callbacks cb 0x5623181a3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9ca5a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54474/0 (socket says 172.21.15.16:54474) 2024-05-27T06:55:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.111+0000 7f9ca5a00700 1 -- 172.21.15.16:0/3291242172 learned_addr learned my addr 172.21.15.16:0/3291242172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5000700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182dbd60 0x5623182f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca6400700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 0x5623182fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 -- 172.21.15.16:0/3291242172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 msgr2=0x5623182fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 0x5623182fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 -- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182dbd60 msgr2=0x5623182f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182dbd60 0x5623182f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 -- 172.21.15.16:0/3291242172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623182ff220 con 0x56231815cdf0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca6400700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 0x5623182fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9ca5a00700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c940a7a60 tx=0x7f9c940227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea4e758fe291b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9c87400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.112+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940c3050 con 0x56231815cdf0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9c940037c0 con 0x56231815cdf0 2024-05-27T06:55:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c940aaef0 con 0x56231815cdf0 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9c940e3040 con 0x56231815cdf0 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 10 client.14709 fetch_fsmap learned FSMap version 17 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562318300480 con 0x56231815cdf0 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c940aaef0 con 0x56231815cdf0 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f9c940aaa50 con 0x56231815cdf0 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.113+0000 7f9cb6e81b80 20 client.14709 populate_metadata read hostname 'smithi016' 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.114+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562318362440 0x5623183638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.114+0000 7f9cb6e81b80 4 client.14709 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.114+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "status"}) -- 0x562318205520 con 0x562318362440 2024-05-27T06:55:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.114+0000 7f9ca5000700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562318362440 0x5623183638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:55:59.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.115+0000 7f9ca5000700 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562318362440 0x5623183638b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5072dbfd87c7a8e5 server_cookie=eec2b636278a14a7 in_seq=0 out_seq=0 2024-05-27T06:55:59.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.115+0000 7f9c87400700 10 client.14709 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9c87400700 1 -- 172.21.15.16:0/3291242172 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562318205520 con 0x562318362440 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9c87400700 10 client.14709 handle_command_reply: tid=0 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 client.14709 shutdown 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 msgr2=0x5623182f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c940a7a60 tx=0x7f9c940227e0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 shutdown_connections 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562318362440 0x5623183638b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623182e1ce0 0x5623182fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56231815cdf0 0x5623182f0120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 --2- 172.21.15.16:0/3291242172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623182dbd60 0x5623182f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 >> 172.21.15.16:0/3291242172 conn(0x5623181bcf70 msgr2=0x5623182df0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.116+0000 7f9c87400700 0 client.14709 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T06:55:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 shutdown_connections 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9cb6e81b80 1 -- 172.21.15.16:0/3291242172 wait complete. 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9cb6e81b80 20 client.14709 trim_cache size 0 max 16384 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34517, 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:55:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 44.721019982999998, 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 70.357979463999996 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 msgr2=0x7f9c880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 0x7f9c880b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c98012780 tx=0x7f9c98007670 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 msgr2=0x7f9ca019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.117+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c940036f0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 shutdown_connections 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9c880b2900 0x7f9c880b4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010c5a0 0x7f9ca01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca0106560 0x7f9ca019cf80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 --2- 172.21.15.16:0/688904983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca0106eb0 0x7f9ca019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 >> 172.21.15.16:0/688904983 conn(0x7f9ca0102040 msgr2=0x7f9ca0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 shutdown_connections 2024-05-27T06:55:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:55:59.118+0000 7f9ca8400700 1 -- 172.21.15.16:0/688904983 wait complete. 2024-05-27T06:55:59.672 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:55:59 smithi069 ceph-mon[35276]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:59.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:55:59 smithi164 ceph-mon[35327]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:55:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:55:59 smithi016 ceph-mon[31415]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:00.010 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T06:56:00.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:00.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T06:56:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:01.667 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:01 smithi016 ceph-mon[31415]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:01 smithi164 ceph-mon[35327]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:01 smithi069 ceph-mon[35276]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.834+0000 7f69a9400700 1 -- 172.21.15.16:0/2442924150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 msgr2=0x7f69a4109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 --2- 172.21.15.16:0/2442924150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a4109300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b0d0 tx=0x7f698c00b3e0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 -- 172.21.15.16:0/2442924150 shutdown_connections 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 --2- 172.21.15.16:0/2442924150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a41111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 --2- 172.21.15.16:0/2442924150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 0x7f69a41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 --2- 172.21.15.16:0/2442924150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a4109300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 -- 172.21.15.16:0/2442924150 >> 172.21.15.16:0/2442924150 conn(0x7f69a40ffe90 msgr2=0x7f69a41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 -- 172.21.15.16:0/2442924150 shutdown_connections 2024-05-27T06:56:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.835+0000 7f69a9400700 1 -- 172.21.15.16:0/2442924150 wait complete. 2024-05-27T06:56:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.836+0000 7f69a9400700 1 Processor -- start 2024-05-27T06:56:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 -- start start 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 0x7f69a419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114300 con 0x7f69a4108ef0 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114180 con 0x7f69a4109840 2024-05-27T06:56:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.837+0000 7f69a9400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114480 con 0x7f69a41045c0 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38246/0 (socket says 172.21.15.16:38246) 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 -- 172.21.15.16:0/2415270251 learned_addr learned my addr 172.21.15.16:0/2415270251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f69a3200700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f69a2800700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 0x7f69a419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 -- 172.21.15.16:0/2415270251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 msgr2=0x7f69a419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 0x7f69a419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 -- 172.21.15.16:0/2415270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 msgr2=0x7f69a419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f6999e00700 1 -- 172.21.15.16:0/2415270251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a419fbe0 con 0x7f69a4108ef0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.838+0000 7f69a3200700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f6999e00700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f698c009850 tx=0x7f698c007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c01f030 con 0x7f69a4108ef0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f698c01bad0 con 0x7f69a4108ef0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c012b20 con 0x7f69a4108ef0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a419fe70 con 0x7f69a4108ef0 2024-05-27T06:56:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.839+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41a77f0 con 0x7f69a4108ef0 2024-05-27T06:56:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.841+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4105a70 con 0x7f69a4108ef0 2024-05-27T06:56:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.843+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f698c022030 con 0x7f69a4108ef0 2024-05-27T06:56:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.844+0000 7f699be00700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.845+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f698c00f020 con 0x7f69a4108ef0 2024-05-27T06:56:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.845+0000 7f69a2800700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.846+0000 7f69a2800700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 0x7f69880b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6994005950 tx=0x7f69940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:02.847+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f698c029020 con 0x7f69a4108ef0 2024-05-27T06:56:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.141+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f69a402e5e0 con 0x7f69a4108ef0 2024-05-27T06:56:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.142+0000 7f699be00700 1 -- 172.21.15.16:0/2415270251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f698c0ab880 con 0x7f69a4108ef0 2024-05-27T06:56:03.134 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T06:56:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.144+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 msgr2=0x7f69880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.144+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 0x7f69880b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6994005950 tx=0x7f69940058e0 comp rx=0 tx=0).stop 2024-05-27T06:56:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.144+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 msgr2=0x7f69a419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.144+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f698c009850 tx=0x7f698c007e40 comp rx=0 tx=0).stop 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 shutdown_connections 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69a4109840 0x7f69a419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69a41045c0 0x7f69a419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 --2- 172.21.15.16:0/2415270251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a4108ef0 0x7f69a419b2b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 >> 172.21.15.16:0/2415270251 conn(0x7f69a40ffe90 msgr2=0x7f69a41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 shutdown_connections 2024-05-27T06:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:03.145+0000 7f69a9400700 1 -- 172.21.15.16:0/2415270251 wait complete. 2024-05-27T06:56:03.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:03 smithi016 ceph-mon[31415]: pgmap v420: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:03.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:03 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:56:03.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:03 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2415270251' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T06:56:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:03 smithi164 ceph-mon[35327]: pgmap v420: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:03 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:56:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:03 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2415270251' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T06:56:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:03 smithi069 ceph-mon[35276]: pgmap v420: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:03 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:56:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:03 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2415270251' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T06:56:03.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T06:56:03.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:03.986 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-05-27T06:56:04.026 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T06:56:04.027 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41767sec preferred_lft 41767sec 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-05-27T06:56:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-05-27T06:56:04.029 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T06:56:04.030 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T06:56:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T06:56:04.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:04.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:04.657 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T06:56:04.723 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-05-27T06:56:04.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:04.726 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-05-27T06:56:04.835 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-05-27T06:56:04.835 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T06:56:04.835 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-27T06:56:04.835 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-27T06:56:04.835 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T06:56:04.836 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T06:56:04.837 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T06:56:04.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-27T06:56:04.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-27T06:56:04.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-27T06:56:04.837 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T06:56:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T06:56:05.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:04 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:04 smithi069 ceph-mon[35276]: pgmap v421: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:05.205 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.205 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.205 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.205 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:04 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.205 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:04 smithi164 ceph-mon[35327]: pgmap v421: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:04 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:04 smithi016 ceph-mon[31415]: pgmap v421: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:05.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:05.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:05.771 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-27T06:56:05.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:05.813 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-05-27T06:56:05.895 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T06:56:05.895 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T06:56:05.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:05.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-27T06:56:05.895 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T06:56:06.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T06:56:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:06.110 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-27T06:56:06.111 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T06:56:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T06:56:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:06.421 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T06:56:06.421 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T06:56:06.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-05-27T06:56:06.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T06:56:06.421 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T06:56:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T06:56:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:06.620 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-27T06:56:06.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T06:56:06.620 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:06.668 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T06:56:06.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T06:56:06.668 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:06.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T06:56:06.835 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T06:56:06.878 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T06:56:06.878 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T06:56:06.879 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T06:56:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T06:56:06.881 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:06.882 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:06.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:06.886 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T06:56:06.974 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T06:56:06.975 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-27T06:56:07.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T06:56:07.095 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T06:56:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T06:56:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T06:56:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T06:56:07.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:07.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:07.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:07.144 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T06:56:07.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-05-27T06:56:07.334+0000 7ff7c7955480 -1 init, newargv = 0x7ff7b0004ba0 newargc=15 2024-05-27T06:56:07.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[76150]: starting ceph client 2024-05-27T06:56:07.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[76150]: starting fuse 2024-05-27T06:56:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: pgmap v422: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:07 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: pgmap v422: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:07 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: pgmap v422: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:07 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:08.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T06:56:08.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:56:08.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:56:08.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:08 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:08.439 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T06:56:08.497 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T06:56:08.498 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T06:56:08.499 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T06:56:08.500 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:08.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:08.501 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T06:56:08.553 INFO:teuthology.orchestra.run.smithi016.stdout:119 2024-05-27T06:56:08.553 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-27T06:56:08.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-05-27T06:56:08.553 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-05-27T06:56:08.553 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-05-27T06:56:08.553 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-05-27T06:56:08.554 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-05-27T06:56:08.555 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-05-27T06:56:08.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:56:08.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:56:08.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:08 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:56:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:56:08.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:08 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:56:08.735 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.76150.asok 2024-05-27T06:56:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:08.754 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76150.asok 2024-05-27T06:56:08.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:08.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76150.asok status 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-27T06:56:09.066 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "76150", 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 14721, 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 14721 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4263978501 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T06:56:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4263978501 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.14721 192.168.0.1:0/4263978501", 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/4263978501", 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 88, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-05-27T06:56:09.068 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:56:09.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:09.089 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-05-27T06:56:09.089 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:09.146 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-05-27T06:56:09.147 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:09.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:09.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:09.245 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-27T06:56:09.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:09 smithi016 ceph-mon[31415]: pgmap v423: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T06:56:09.418 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-05-27T06:56:09.418 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-27T06:56:09.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:56:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T06:56:09.429 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:56:09.430 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T06:56:09.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T06:56:09.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:09 smithi164 ceph-mon[35327]: pgmap v423: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:09 smithi069 ceph-mon[35276]: pgmap v423: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:11.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:11 smithi016 ceph-mon[31415]: pgmap v424: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:11.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:11 smithi164 ceph-mon[35327]: pgmap v424: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:11 smithi069 ceph-mon[35276]: pgmap v424: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 -- 172.21.15.16:0/95560722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc075b50 msgr2=0x7efecc075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 --2- 172.21.15.16:0/95560722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc075b50 0x7efecc075f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efebc009990 tx=0x7efebc009ca0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 -- 172.21.15.16:0/95560722 shutdown_connections 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 --2- 172.21.15.16:0/95560722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc115360 0x7efecc117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 --2- 172.21.15.16:0/95560722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc075b50 0x7efecc075f60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 --2- 172.21.15.16:0/95560722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc0764a0 0x7efecc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 -- 172.21.15.16:0/95560722 >> 172.21.15.16:0/95560722 conn(0x7efecc100030 msgr2=0x7efecc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 -- 172.21.15.16:0/95560722 shutdown_connections 2024-05-27T06:56:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.371+0000 7efed2200700 1 -- 172.21.15.16:0/95560722 wait complete. 2024-05-27T06:56:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.372+0000 7efed2200700 1 Processor -- start 2024-05-27T06:56:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 -- start start 2024-05-27T06:56:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc075b50 0x7efecc1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 0x7efecc1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11a960 con 0x7efecc115360 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11a7e0 con 0x7efecc075b50 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efed2200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11aae0 con 0x7efecc0764a0 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.373+0000 7efecbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57224/0 (socket says 172.21.15.16:57224) 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 -- 172.21.15.16:0/3344760529 learned_addr learned my addr 172.21.15.16:0/3344760529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecb400700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc075b50 0x7efecc1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efec2a00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 0x7efecc1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 -- 172.21.15.16:0/3344760529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 msgr2=0x7efecc1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 0x7efecc1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 -- 172.21.15.16:0/3344760529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc075b50 msgr2=0x7efecc1a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc075b50 0x7efecc1a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efecbe00700 1 -- 172.21.15.16:0/3344760529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1a6170 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.374+0000 7efec2a00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 0x7efecc1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efecbe00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efec4007d70 tx=0x7efec400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4016020 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec400ad40 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec40115e0 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc1a6460 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.375+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1a6950 con 0x7efecc115360 2024-05-27T06:56:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.376+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc0512b0 con 0x7efecc115360 2024-05-27T06:56:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.381+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efec4019020 con 0x7efecc115360 2024-05-27T06:56:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.383+0000 7efec8c00700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 0x7efeb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.383+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7efec40e29e0 con 0x7efecc115360 2024-05-27T06:56:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.383+0000 7efecb400700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 0x7efeb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.384+0000 7efecb400700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 0x7efeb00b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efebc009750 tx=0x7efebc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:12.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.385+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efec4020020 con 0x7efecc115360 2024-05-27T06:56:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.643+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efecc02e5e0 con 0x7efecc115360 2024-05-27T06:56:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.644+0000 7efec8c00700 1 -- 172.21.15.16:0/3344760529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+592 (secure 0 0 0) 0x7efec40aa340 con 0x7efecc115360 2024-05-27T06:56:12.636 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3562581452 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:12.636 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1801942445 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:12.636 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2589421123 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:12.636 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3708593763 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3327639895 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4243270602 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:12.637 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 msgr2=0x7efeb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 0x7efeb00b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efebc009750 tx=0x7efebc005fd0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 msgr2=0x7efecc1a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efec4007d70 tx=0x7efec400a340 comp rx=0 tx=0).stop 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 shutdown_connections 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efeb00b2880 0x7efeb00b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc115360 0x7efecc1a5b80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efecc075b50 0x7efecc1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 --2- 172.21.15.16:0/3344760529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0764a0 0x7efecc1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 >> 172.21.15.16:0/3344760529 conn(0x7efecc100030 msgr2=0x7efecc100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.648+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 shutdown_connections 2024-05-27T06:56:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:12.649+0000 7efed2200700 1 -- 172.21.15.16:0/3344760529 wait complete. 2024-05-27T06:56:12.642 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-05-27T06:56:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:13 smithi016 ceph-mon[31415]: pgmap v425: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:13 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3344760529' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:13.511 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:56:13.511 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T06:56:13.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T06:56:13.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:13 smithi164 ceph-mon[35327]: pgmap v425: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:13.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:13 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3344760529' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:13 smithi069 ceph-mon[35276]: pgmap v425: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T06:56:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:13 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3344760529' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:14.061 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:15 smithi016 ceph-mon[31415]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:15.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:15 smithi164 ceph-mon[35327]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:15.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:15 smithi069 ceph-mon[35276]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 -- 172.21.15.16:0/2019330158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 msgr2=0x7fafb4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 --2- 172.21.15.16:0/2019330158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb4109320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099c0 tx=0x7fafa0009cd0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 -- 172.21.15.16:0/2019330158 shutdown_connections 2024-05-27T06:56:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 --2- 172.21.15.16:0/2019330158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb4109860 0x7fafb41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 --2- 172.21.15.16:0/2019330158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb41045e0 0x7fafb41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 --2- 172.21.15.16:0/2019330158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb4109320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 -- 172.21.15.16:0/2019330158 >> 172.21.15.16:0/2019330158 conn(0x7fafb4100030 msgr2=0x7fafb4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.426+0000 7fafb9600700 1 -- 172.21.15.16:0/2019330158 shutdown_connections 2024-05-27T06:56:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.427+0000 7fafb9600700 1 -- 172.21.15.16:0/2019330158 wait complete. 2024-05-27T06:56:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.428+0000 7fafb9600700 1 Processor -- start 2024-05-27T06:56:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 -- start start 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb41045e0 0x7fafb419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb4109860 0x7fafb41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4114370 con 0x7fafb4108f10 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41141f0 con 0x7fafb4109860 2024-05-27T06:56:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.429+0000 7fafb9600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41144f0 con 0x7fafb41045e0 2024-05-27T06:56:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57246/0 (socket says 172.21.15.16:57246) 2024-05-27T06:56:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 -- 172.21.15.16:0/686459312 learned_addr learned my addr 172.21.15.16:0/686459312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb3400700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb4109860 0x7fafb41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2a00700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb41045e0 0x7fafb419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 -- 172.21.15.16:0/686459312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb41045e0 msgr2=0x7fafb419d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb41045e0 0x7fafb419d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 -- 172.21.15.16:0/686459312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb4109860 msgr2=0x7fafb41a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb4109860 0x7fafb41a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.430+0000 7fafb2000700 1 -- 172.21.15.16:0/686459312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb41a1f20 con 0x7fafb4108f10 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafb2000700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b580 tx=0x7fafa000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0027070 con 0x7fafb4108f10 2024-05-27T06:56:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb41a21b0 con 0x7fafb4108f10 2024-05-27T06:56:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa000fda0 con 0x7fafb4108f10 2024-05-27T06:56:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00173f0 con 0x7fafb4108f10 2024-05-27T06:56:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.431+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41a9b30 con 0x7fafb4108f10 2024-05-27T06:56:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.433+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb40512b0 con 0x7fafb4108f10 2024-05-27T06:56:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.435+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fafa002a070 con 0x7fafb4108f10 2024-05-27T06:56:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.436+0000 7fafab400700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 0x7faf980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.436+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fafa00e3590 con 0x7fafb4108f10 2024-05-27T06:56:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.437+0000 7fafb2a00700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 0x7faf980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.438+0000 7fafb2a00700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 0x7faf980b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafa4005950 tx=0x7fafa40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:16.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.442+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fafa0013070 con 0x7fafb4108f10 2024-05-27T06:56:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.697+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafb419e3b0 con 0x7fafb4108f10 2024-05-27T06:56:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.698+0000 7fafab400700 1 -- 172.21.15.16:0/686459312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+592 (secure 0 0 0) 0x7fafa00aaf80 con 0x7fafb4108f10 2024-05-27T06:56:16.694 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3562581452 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:16.694 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1801942445 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:16.694 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2589421123 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3708593763 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3327639895 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4243270602 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T06:56:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 msgr2=0x7faf980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 0x7faf980b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafa4005950 tx=0x7fafa40058e0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 msgr2=0x7fafb419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b580 tx=0x7fafa000f460 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 shutdown_connections 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7faf980b2900 0x7faf980b4e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb4109860 0x7fafb41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafb41045e0 0x7fafb419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 --2- 172.21.15.16:0/686459312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4108f10 0x7fafb419d640 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 >> 172.21.15.16:0/686459312 conn(0x7fafb4100030 msgr2=0x7fafb410a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.702+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 shutdown_connections 2024-05-27T06:56:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:16.703+0000 7fafb9600700 1 -- 172.21.15.16:0/686459312 wait complete. 2024-05-27T06:56:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-05-27T06:56:17.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:17 smithi016 ceph-mon[31415]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:17.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:17 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/686459312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:17.558 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-05-27T06:56:17.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:17.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-27T06:56:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:17 smithi164 ceph-mon[35327]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:17 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/686459312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:17.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:17 smithi069 ceph-mon[35276]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T06:56:17.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:17 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/686459312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T06:56:17.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[76150]: fuse finished with error 0 and tester_r 0 2024-05-27T06:56:17.658 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T06:56:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:19 smithi164 ceph-mon[35327]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T06:56:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:19 smithi016 ceph-mon[31415]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T06:56:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:19 smithi069 ceph-mon[35276]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T06:56:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:21 smithi164 ceph-mon[35327]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:21 smithi016 ceph-mon[31415]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:21.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:21 smithi069 ceph-mon[35276]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:23.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:23 smithi164 ceph-mon[35327]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:23.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:23 smithi016 ceph-mon[31415]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:23.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:23 smithi069 ceph-mon[35276]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:23.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T06:56:23.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T06:56:23.761 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-27T06:56:23.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:56:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:25.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:25 smithi164 ceph-mon[35327]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:25.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:25 smithi016 ceph-mon[31415]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:25 smithi069 ceph-mon[35276]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T06:56:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.822+0000 7f9365400700 1 -- 172.21.15.16:0/3051935117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010b2d0 msgr2=0x7f9360117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.822+0000 7f9365400700 1 --2- 172.21.15.16:0/3051935117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010b2d0 0x7f9360117800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f935800b0a0 tx=0x7f935800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 -- 172.21.15.16:0/3051935117 shutdown_connections 2024-05-27T06:56:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 --2- 172.21.15.16:0/3051935117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010b2d0 0x7f9360117800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 --2- 172.21.15.16:0/3051935117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360110960 0x7f9360110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 --2- 172.21.15.16:0/3051935117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936010a980 0x7f936010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 -- 172.21.15.16:0/3051935117 >> 172.21.15.16:0/3051935117 conn(0x7f9360077c70 msgr2=0x7f93600780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 -- 172.21.15.16:0/3051935117 shutdown_connections 2024-05-27T06:56:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.823+0000 7f9365400700 1 -- 172.21.15.16:0/3051935117 wait complete. 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.824+0000 7f9365400700 1 Processor -- start 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.824+0000 7f9365400700 1 -- start start 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f936010b2d0 0x7f93601a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9360110960 0x7f93601a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f936011a930 con 0x7f936010a980 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f936011a7b0 con 0x7f936010b2d0 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f9365400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f936011aab0 con 0x7f9360110960 2024-05-27T06:56:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f935e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f935e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50642/0 (socket says 172.21.15.16:50642) 2024-05-27T06:56:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f935e800700 1 -- 172.21.15.16:0/284123819 learned_addr learned my addr 172.21.15.16:0/284123819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.825+0000 7f935de00700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f936010b2d0 0x7f93601a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935f200700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9360110960 0x7f93601a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 -- 172.21.15.16:0/284123819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9360110960 msgr2=0x7f93601a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9360110960 0x7f93601a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 -- 172.21.15.16:0/284123819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f936010b2d0 msgr2=0x7f93601a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f936010b2d0 0x7f93601a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 -- 172.21.15.16:0/284123819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93601a61e0 con 0x7f936010a980 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.826+0000 7f935e800700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f935000c8f0 tx=0x7f935000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.827+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93500078d0 con 0x7f936010a980 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.827+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f935000f450 con 0x7f936010a980 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.827+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350018660 con 0x7f936010a980 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.827+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93601a6470 con 0x7f936010a980 2024-05-27T06:56:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.827+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93601a6a00 con 0x7f936010a980 2024-05-27T06:56:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.828+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93600512b0 con 0x7f936010a980 2024-05-27T06:56:26.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.831+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f935000f900 con 0x7f936010a980 2024-05-27T06:56:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.832+0000 7f9357400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 0x7f93440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.833+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f93500e23a0 con 0x7f936010a980 2024-05-27T06:56:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.833+0000 7f935de00700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 0x7f93440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.834+0000 7f935de00700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 0x7f93440b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9360109940 tx=0x7f934c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:26.835+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9350014030 con 0x7f936010a980 2024-05-27T06:56:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.072+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f936002e5e0 con 0x7f936010a980 2024-05-27T06:56:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.074+0000 7f9357400700 1 -- 172.21.15.16:0/284123819 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24566 (secure 0 0 0) 0x7f93500a9e10 con 0x7f936010a980 2024-05-27T06:56:27.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:56:27.068 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:55:39.287742+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 msgr2=0x7f93440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 0x7f93440b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9360109940 tx=0x7f934c00a380 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 msgr2=0x7f93601a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f935000c8f0 tx=0x7f935000cc00 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 shutdown_connections 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f93440b2900 0x7f93440b4e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9360110960 0x7f93601a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936010a980 0x7f93601a13c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 --2- 172.21.15.16:0/284123819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f936010b2d0 0x7f93601a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 >> 172.21.15.16:0/284123819 conn(0x7f9360077c70 msgr2=0x7f936007b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 shutdown_connections 2024-05-27T06:56:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:27.078+0000 7f9365400700 1 -- 172.21.15.16:0/284123819 wait complete. 2024-05-27T06:56:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:27 smithi016 ceph-mon[31415]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:27 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/284123819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:27 smithi164 ceph-mon[35327]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:27 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/284123819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:27 smithi069 ceph-mon[35276]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:27 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/284123819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:27.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:56:28.471 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:29.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:29 smithi016 ceph-mon[31415]: pgmap v433: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:29 smithi164 ceph-mon[35327]: pgmap v433: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:29 smithi069 ceph-mon[35276]: pgmap v433: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.879+0000 7fdf63400700 1 -- 172.21.15.16:0/967594597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 msgr2=0x7fdf5c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.879+0000 7fdf63400700 1 --2- 172.21.15.16:0/967594597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c113350 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400b0a0 tx=0x7fdf5400b3b0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.879+0000 7fdf63400700 1 -- 172.21.15.16:0/967594597 shutdown_connections 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.879+0000 7fdf63400700 1 --2- 172.21.15.16:0/967594597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c113350 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.880+0000 7fdf63400700 1 --2- 172.21.15.16:0/967594597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.880+0000 7fdf63400700 1 --2- 172.21.15.16:0/967594597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.880+0000 7fdf63400700 1 -- 172.21.15.16:0/967594597 >> 172.21.15.16:0/967594597 conn(0x7fdf5c102070 msgr2=0x7fdf5c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.880+0000 7fdf63400700 1 -- 172.21.15.16:0/967594597 shutdown_connections 2024-05-27T06:56:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.880+0000 7fdf63400700 1 -- 172.21.15.16:0/967594597 wait complete. 2024-05-27T06:56:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.881+0000 7fdf63400700 1 Processor -- start 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.881+0000 7fdf63400700 1 -- start start 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.881+0000 7fdf63400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf63400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf63400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf63400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c1165e0 con 0x7fdf5c106ec0 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf63400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c116460 con 0x7fdf5c10c5b0 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf63400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c116760 con 0x7fdf5c106570 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf5be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf5be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50662/0 (socket says 172.21.15.16:50662) 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf5be00700 1 -- 172.21.15.16:0/3525918669 learned_addr learned my addr 172.21.15.16:0/3525918669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf60a00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.882+0000 7fdf61400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 -- 172.21.15.16:0/3525918669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 msgr2=0x7fdf5c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 -- 172.21.15.16:0/3525918669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 msgr2=0x7fdf5c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 -- 172.21.15.16:0/3525918669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c1a1dd0 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf60a00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf61400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf5be00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c9b0 tx=0x7fdf5000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50007aa0 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c1a20c0 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf50007c40 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a9a90 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.883+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50018600 con 0x7fdf5c106ec0 2024-05-27T06:56:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.885+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0512b0 con 0x7fdf5c106ec0 2024-05-27T06:56:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.887+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdf5000f450 con 0x7fdf5c106ec0 2024-05-27T06:56:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.889+0000 7fdf59600700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 0x7fdf440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.889+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fdf500e2770 con 0x7fdf5c106ec0 2024-05-27T06:56:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.889+0000 7fdf60a00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 0x7fdf440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:30.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.890+0000 7fdf60a00700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 0x7fdf440b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c005e10 tx=0x7fdf4c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:30.892+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf50054df0 con 0x7fdf5c106ec0 2024-05-27T06:56:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.132+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf5c19e310 con 0x7fdf5c106ec0 2024-05-27T06:56:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.134+0000 7fdf59600700 1 -- 172.21.15.16:0/3525918669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24566 (secure 0 0 0) 0x7fdf50014030 con 0x7fdf5c106ec0 2024-05-27T06:56:31.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:56:31.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:55:39.287742+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 msgr2=0x7fdf440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 0x7fdf440b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c005e10 tx=0x7fdf4c005d60 comp rx=0 tx=0).stop 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 msgr2=0x7fdf5c19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c9b0 tx=0x7fdf5000cd70 comp rx=0 tx=0).stop 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 shutdown_connections 2024-05-27T06:56:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf440b2880 0x7fdf440b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf5c10c5b0 0x7fdf5c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5c106570 0x7fdf5c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 --2- 172.21.15.16:0/3525918669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf5c106ec0 0x7fdf5c19d550 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 >> 172.21.15.16:0/3525918669 conn(0x7fdf5c102070 msgr2=0x7fdf5c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.137+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 shutdown_connections 2024-05-27T06:56:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:31.138+0000 7fdf63400700 1 -- 172.21.15.16:0/3525918669 wait complete. 2024-05-27T06:56:31.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:31 smithi016 ceph-mon[31415]: pgmap v434: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:31.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:31 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3525918669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:31 smithi164 ceph-mon[35327]: pgmap v434: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:31 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3525918669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:31 smithi069 ceph-mon[35276]: pgmap v434: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:31 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3525918669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:56:31.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-27T06:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:33 smithi164 ceph-mon[35327]: pgmap v435: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:33 smithi016 ceph-mon[31415]: pgmap v435: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:33.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:33 smithi069 ceph-mon[35276]: pgmap v435: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 -- 172.21.15.16:0/491053623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 msgr2=0x7fc9c0113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 --2- 172.21.15.16:0/491053623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c0113340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800b0a0 tx=0x7fc9b800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 -- 172.21.15.16:0/491053623 shutdown_connections 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 --2- 172.21.15.16:0/491053623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c0113340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 --2- 172.21.15.16:0/491053623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 0x7fc9c010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 --2- 172.21.15.16:0/491053623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 0x7fc9c0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 -- 172.21.15.16:0/491053623 >> 172.21.15.16:0/491053623 conn(0x7fc9c0102040 msgr2=0x7fc9c0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 -- 172.21.15.16:0/491053623 shutdown_connections 2024-05-27T06:56:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.810+0000 7fc9c6200700 1 -- 172.21.15.16:0/491053623 wait complete. 2024-05-27T06:56:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.812+0000 7fc9c6200700 1 Processor -- start 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.812+0000 7fc9c6200700 1 -- start start 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.812+0000 7fc9c6200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 0x7fc9c019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.812+0000 7fc9c6200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9c6200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 0x7fc9c01a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9c6200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01165d0 con 0x7fc9c0106eb0 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9c6200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0116450 con 0x7fc9c0106560 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9c6200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0116750 con 0x7fc9c010c5a0 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9bea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9bea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33936/0 (socket says 172.21.15.16:33936) 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9bea00700 1 -- 172.21.15.16:0/1096811672 learned_addr learned my addr 172.21.15.16:0/1096811672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9bf400700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 0x7fc9c019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.813+0000 7fc9bfe00700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 0x7fc9c01a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 -- 172.21.15.16:0/1096811672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 msgr2=0x7fc9c01a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 0x7fc9c01a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 -- 172.21.15.16:0/1096811672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 msgr2=0x7fc9c019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 0x7fc9c019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 -- 172.21.15.16:0/1096811672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c01a1d20 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9bea00700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c8c0 tx=0x7fc9ac00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac004220 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9ac0043c0 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c01a2010 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac005660 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.814+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c01a2550 con 0x7fc9c0106eb0 2024-05-27T06:56:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.816+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c00512b0 con 0x7fc9c0106eb0 2024-05-27T06:56:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.821+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc9ac004750 con 0x7fc9c0106eb0 2024-05-27T06:56:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.823+0000 7fc9b7e00700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 0x7fc9a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:34.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.823+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc9ac0e2610 con 0x7fc9c0106eb0 2024-05-27T06:56:34.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.823+0000 7fc9bf400700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 0x7fc9a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:34.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.824+0000 7fc9bf400700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 0x7fc9a40b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9c01079d0 tx=0x7fc9b000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:34.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:34.825+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9ac014030 con 0x7fc9c0106eb0 2024-05-27T06:56:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.114+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc9c002e5e0 con 0x7fc9c0106eb0 2024-05-27T06:56:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.275+0000 7fc9b7e00700 1 -- 172.21.15.16:0/1096811672 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fc9ac0a9f70 con 0x7fc9c0106eb0 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 msgr2=0x7fc9a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 0x7fc9a40b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9c01079d0 tx=0x7fc9b000a380 comp rx=0 tx=0).stop 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 msgr2=0x7fc9c019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c8c0 tx=0x7fc9ac00cbd0 comp rx=0 tx=0).stop 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 shutdown_connections 2024-05-27T06:56:35.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc9a40b2880 0x7fc9a40b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9c010c5a0 0x7fc9c01a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9c0106560 0x7fc9c019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 --2- 172.21.15.16:0/1096811672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c0106eb0 0x7fc9c019d4a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.279+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 >> 172.21.15.16:0/1096811672 conn(0x7fc9c0102040 msgr2=0x7fc9c01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.280+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 shutdown_connections 2024-05-27T06:56:35.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:35.280+0000 7fc9c6200700 1 -- 172.21.15.16:0/1096811672 wait complete. 2024-05-27T06:56:35.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:35 smithi016 ceph-mon[31415]: pgmap v436: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:35.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:35 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T06:56:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:35 smithi164 ceph-mon[35327]: pgmap v436: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:35 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T06:56:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:35 smithi069 ceph-mon[35276]: pgmap v436: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:35 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T06:56:36.194 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-27T06:56:36.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-27T06:56:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:36 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T06:56:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:36 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:36.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:36 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T06:56:36.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:36 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:36.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:36 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1096811672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T06:56:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:36 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:37 smithi164 ceph-mon[35327]: pgmap v437: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:37.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:37 smithi016 ceph-mon[31415]: pgmap v437: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:37.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:37 smithi069 ceph-mon[35276]: pgmap v437: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 -- 172.21.15.16:0/3796589889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f4810b860 msgr2=0x7f8f481131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 --2- 172.21.15.16:0/3796589889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f4810b860 0x7f8f481131c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b0d0 tx=0x7f8f4000b3e0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 -- 172.21.15.16:0/3796589889 shutdown_connections 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 --2- 172.21.15.16:0/3796589889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f4810b860 0x7f8f481131c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 --2- 172.21.15.16:0/3796589889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f48106630 0x7f8f48106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 --2- 172.21.15.16:0/3796589889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 0x7f8f4810b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 -- 172.21.15.16:0/3796589889 >> 172.21.15.16:0/3796589889 conn(0x7f8f48102020 msgr2=0x7f8f48104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 -- 172.21.15.16:0/3796589889 shutdown_connections 2024-05-27T06:56:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.922+0000 7f8f4d200700 1 -- 172.21.15.16:0/3796589889 wait complete. 2024-05-27T06:56:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.923+0000 7f8f4d200700 1 Processor -- start 2024-05-27T06:56:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.923+0000 7f8f4d200700 1 -- start start 2024-05-27T06:56:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 0x7f8f4819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f4810b860 0x7f8f481a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48116390 con 0x7f8f48106630 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48116210 con 0x7f8f4810b860 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f4d200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48116510 con 0x7f8f4810af10 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f47400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f47400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33952/0 (socket says 172.21.15.16:33952) 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.924+0000 7f8f47400700 1 -- 172.21.15.16:0/4265268898 learned_addr learned my addr 172.21.15.16:0/4265268898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47e00700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f4810b860 0x7f8f481a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f46a00700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 0x7f8f4819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 -- 172.21.15.16:0/4265268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 msgr2=0x7f8f4819d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 0x7f8f4819d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 -- 172.21.15.16:0/4265268898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f4810b860 msgr2=0x7f8f481a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f4810b860 0x7f8f481a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 -- 172.21.15.16:0/4265268898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f481a1bc0 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.925+0000 7f8f47400700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800d950 tx=0x7f8f3800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.926+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38016070 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.926+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f38010460 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.926+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3800f620 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.926+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f481a1eb0 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.926+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a9880 con 0x7f8f48106630 2024-05-27T06:56:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.927+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f480512b0 con 0x7f8f48106630 2024-05-27T06:56:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.930+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8f38019070 con 0x7f8f48106630 2024-05-27T06:56:38.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.932+0000 7f8f3fe00700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 0x7f8f2c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:38.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.932+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f8f380e2160 con 0x7f8f48106630 2024-05-27T06:56:38.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.933+0000 7f8f46a00700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 0x7f8f2c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:38.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.933+0000 7f8f46a00700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 0x7f8f2c0b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f34005fd0 tx=0x7f8f3400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:38.938+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f380a9ac0 con 0x7f8f48106630 2024-05-27T06:56:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.235+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f8f4819e100 con 0x7f8f48106630 2024-05-27T06:56:39.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.326+0000 7f8f3fe00700 1 -- 172.21.15.16:0/4265268898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f8f380a93c0 con 0x7f8f48106630 2024-05-27T06:56:39.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.331+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 msgr2=0x7f8f2c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:39.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.331+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 0x7f8f2c0b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f34005fd0 tx=0x7f8f3400af20 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.331+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 msgr2=0x7f8f4819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.331+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800d950 tx=0x7f8f3800dd10 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 shutdown_connections 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f8f2c0b2900 0x7f8f2c0b4e10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f4810b860 0x7f8f481a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48106630 0x7f8f4819cda0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 --2- 172.21.15.16:0/4265268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f4810af10 0x7f8f4819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 >> 172.21.15.16:0/4265268898 conn(0x7f8f48102020 msgr2=0x7f8f4810a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.332+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 shutdown_connections 2024-05-27T06:56:39.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:39.333+0000 7f8f4d200700 1 -- 172.21.15.16:0/4265268898 wait complete. 2024-05-27T06:56:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:39 smithi164 ceph-mon[35327]: pgmap v438: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-27T06:56:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-27T06:56:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:39 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-05-27T06:56:39.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:39 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T06:56:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:39 smithi016 ceph-mon[31415]: pgmap v438: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-27T06:56:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-27T06:56:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:39 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-05-27T06:56:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:39 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T06:56:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:39 smithi069 ceph-mon[35276]: pgmap v438: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T06:56:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-27T06:56:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-27T06:56:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:39 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-05-27T06:56:39.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:39 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T06:56:40.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-27T06:56:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T06:56:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T06:56:40.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:40 smithi164 ceph-mon[35327]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T06:56:40.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T06:56:40.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:40 smithi016 ceph-mon[31415]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T06:56:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T06:56:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4265268898' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T06:56:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-27T06:56:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T06:56:40.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:40 smithi069 ceph-mon[35276]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T06:56:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: pgmap v440: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T06:56:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T06:56:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:active 2024-05-27T06:56:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T06:56:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T06:56:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T06:56:41.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:41 smithi164 ceph-mon[35327]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T06:56:41.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: pgmap v440: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:active 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T06:56:41.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:41 smithi016 ceph-mon[31415]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: pgmap v440: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] up:active 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T06:56:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:41 smithi069 ceph-mon[35276]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T06:56:42.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:42 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:active 2024-05-27T06:56:42.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:42 smithi016 ceph-mon[31415]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T06:56:42.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:42 smithi016 ceph-mon[31415]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T06:56:42.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:42 smithi016 ceph-mon[31415]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T06:56:42.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:42 smithi016 ceph-mon[31415]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T06:56:42.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:42 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:active 2024-05-27T06:56:42.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:42 smithi164 ceph-mon[35327]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T06:56:42.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:42 smithi164 ceph-mon[35327]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T06:56:42.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:42 smithi164 ceph-mon[35327]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T06:56:42.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:42 smithi164 ceph-mon[35327]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T06:56:42.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:42 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] up:active 2024-05-27T06:56:42.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:42 smithi069 ceph-mon[35276]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T06:56:42.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:42 smithi069 ceph-mon[35276]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T06:56:42.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:42 smithi069 ceph-mon[35276]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T06:56:42.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:42 smithi069 ceph-mon[35276]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T06:56:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 -- 172.21.15.16:0/1148104775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c108f10 msgr2=0x7f6a6c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 --2- 172.21.15.16:0/1148104775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c108f10 0x7f6a6c109320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6a540099c0 tx=0x7f6a54009cd0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 -- 172.21.15.16:0/1148104775 shutdown_connections 2024-05-27T06:56:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 --2- 172.21.15.16:0/1148104775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c109860 0x7f6a6c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 --2- 172.21.15.16:0/1148104775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 --2- 172.21.15.16:0/1148104775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c108f10 0x7f6a6c109320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.181+0000 7f6a72000700 1 -- 172.21.15.16:0/1148104775 >> 172.21.15.16:0/1148104775 conn(0x7f6a6c100030 msgr2=0x7f6a6c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.182+0000 7f6a72000700 1 -- 172.21.15.16:0/1148104775 shutdown_connections 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.182+0000 7f6a72000700 1 -- 172.21.15.16:0/1148104775 wait complete. 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 Processor -- start 2024-05-27T06:56:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 -- start start 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 0x7f6a6c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c109860 0x7f6a6c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c114370 con 0x7f6a6c1045e0 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1141f0 con 0x7f6a6c108f10 2024-05-27T06:56:43.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.183+0000 7f6a72000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1144f0 con 0x7f6a6c109860 2024-05-27T06:56:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40304/0 (socket says 172.21.15.16:40304) 2024-05-27T06:56:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 -- 172.21.15.16:0/4115266920 learned_addr learned my addr 172.21.15.16:0/4115266920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a62a00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 0x7f6a6c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6be00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c109860 0x7f6a6c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 -- 172.21.15.16:0/4115266920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c109860 msgr2=0x7f6a6c1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c109860 0x7f6a6c1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 -- 172.21.15.16:0/4115266920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 msgr2=0x7f6a6c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 0x7f6a6c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a6b400700 1 -- 172.21.15.16:0/4115266920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c1a1e90 con 0x7f6a6c1045e0 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.184+0000 7f6a62a00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 0x7f6a6c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:43.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a6b400700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c920 tx=0x7f6a5c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c007b00 con 0x7f6a6c1045e0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a5c004cc0 con 0x7f6a6c1045e0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c1a2180 con 0x7f6a6c1045e0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0054f0 con 0x7f6a6c1045e0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.185+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1a9aa0 con 0x7f6a6c1045e0 2024-05-27T06:56:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.187+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6c0512b0 con 0x7f6a6c1045e0 2024-05-27T06:56:43.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.189+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6a5c007ca0 con 0x7f6a6c1045e0 2024-05-27T06:56:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.190+0000 7f6a68c00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 0x7f6a500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.191+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6a5c0e2260 con 0x7f6a6c1045e0 2024-05-27T06:56:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.191+0000 7f6a62a00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 0x7f6a500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.192+0000 7f6a62a00700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 0x7f6a500b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009990 tx=0x7f6a54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.195+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a5c014030 con 0x7f6a6c1045e0 2024-05-27T06:56:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:43.491+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6a6c1a9c70 con 0x7f6a6c1045e0 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: pgmap v442: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 5 op/s 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:active 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: Cluster is now healthy 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T06:56:43.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T06:56:43.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:43 smithi016 ceph-mon[31415]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T06:56:43.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: pgmap v442: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 5 op/s 2024-05-27T06:56:43.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:active 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: Cluster is now healthy 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T06:56:43.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:43 smithi164 ceph-mon[35327]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: pgmap v442: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 5 op/s 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] up:active 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: Cluster is now healthy 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T06:56:43.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:43 smithi069 ceph-mon[35276]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T06:56:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.422+0000 7f6a68c00700 1 -- 172.21.15.16:0/4115266920 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f6a5c0a9b70 con 0x7f6a6c1045e0 2024-05-27T06:56:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.426+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 msgr2=0x7f6a500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.426+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 0x7f6a500b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009990 tx=0x7f6a54005fb0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.426+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 msgr2=0x7f6a6c19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.426+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c920 tx=0x7f6a5c00cce0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 shutdown_connections 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6a500b2880 0x7f6a500b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6c109860 0x7f6a6c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c1045e0 0x7f6a6c19cfc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 --2- 172.21.15.16:0/4115266920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a6c108f10 0x7f6a6c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 >> 172.21.15.16:0/4115266920 conn(0x7f6a6c100030 msgr2=0x7f6a6c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 shutdown_connections 2024-05-27T06:56:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:44.427+0000 7f6a72000700 1 -- 172.21.15.16:0/4115266920 wait complete. 2024-05-27T06:56:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:active 2024-05-27T06:56:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-mon[35276]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:44.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T06:56:44.619 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T06:56:44.283+0000 7f0ac5400700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T06:56:44.619 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T06:56:44.283+0000 7f0ac5400700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-27T06:56:44.620 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 06:56:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T06:56:44.283+0000 7f0ac5400700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-27T06:56:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:44 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:active 2024-05-27T06:56:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:44 smithi164 ceph-mon[35327]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T06:56:44.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:44 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] up:active 2024-05-27T06:56:44.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:44 smithi016 ceph-mon[31415]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:44.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T06:56:45.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-27T06:56:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:45 smithi016 ceph-mon[31415]: pgmap v443: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 12 op/s 2024-05-27T06:56:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:45 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T06:56:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:45 smithi016 ceph-mon[31415]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:45 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:45 smithi164 ceph-mon[35327]: pgmap v443: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 12 op/s 2024-05-27T06:56:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:45 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T06:56:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:45 smithi164 ceph-mon[35327]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:45 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:45.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:45 smithi069 ceph-mon[35276]: pgmap v443: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 12 op/s 2024-05-27T06:56:45.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:45 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4115266920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T06:56:45.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:45 smithi069 ceph-mon[35276]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T06:56:45.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:45 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:46 smithi016 ceph-mon[31415]: pgmap v444: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-05-27T06:56:46.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:46 smithi164 ceph-mon[35327]: pgmap v444: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-05-27T06:56:46.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:46 smithi069 ceph-mon[35276]: pgmap v444: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-05-27T06:56:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 -- 172.21.15.16:0/230480473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440075af0 msgr2=0x7f5440075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 --2- 172.21.15.16:0/230480473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440075af0 0x7f5440075f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5430009990 tx=0x7f5430009ca0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 -- 172.21.15.16:0/230480473 shutdown_connections 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 --2- 172.21.15.16:0/230480473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440110bf0 0x7f5440113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 --2- 172.21.15.16:0/230480473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440075af0 0x7f5440075f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 --2- 172.21.15.16:0/230480473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440076440 0x7f54401106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 -- 172.21.15.16:0/230480473 >> 172.21.15.16:0/230480473 conn(0x7f5440100030 msgr2=0x7f5440102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 -- 172.21.15.16:0/230480473 shutdown_connections 2024-05-27T06:56:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.553+0000 7f5445000700 1 -- 172.21.15.16:0/230480473 wait complete. 2024-05-27T06:56:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 Processor -- start 2024-05-27T06:56:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 -- start start 2024-05-27T06:56:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 0x7f544019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440076440 0x7f544019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54401165d0 con 0x7f5440110bf0 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5440116450 con 0x7f5440075af0 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.555+0000 7f5445000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5440116750 con 0x7f5440076440 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40334/0 (socket says 172.21.15.16:40334) 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 -- 172.21.15.16:0/751992115 learned_addr learned my addr 172.21.15.16:0/751992115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543de00700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 0x7f544019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543d400700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440076440 0x7f544019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 -- 172.21.15.16:0/751992115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440076440 msgr2=0x7f544019d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440076440 0x7f544019d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 -- 172.21.15.16:0/751992115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 msgr2=0x7f544019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 0x7f544019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543e800700 1 -- 172.21.15.16:0/751992115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a1e50 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.556+0000 7f543de00700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 0x7f544019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f543e800700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54380094f0 tx=0x7f5438007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438004db0 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a2140 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5438005020 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438015660 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.557+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a9a60 con 0x7f5440110bf0 2024-05-27T06:56:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.558+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54400512b0 con 0x7f5440110bf0 2024-05-27T06:56:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.561+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f54380051d0 con 0x7f5440110bf0 2024-05-27T06:56:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.564+0000 7f5436a00700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 0x7f54240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.564+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f54380e2820 con 0x7f5440110bf0 2024-05-27T06:56:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.564+0000 7f543de00700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 0x7f54240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.565+0000 7f543de00700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 0x7f54240b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f543000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:48.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.567+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5438010070 con 0x7f5440110bf0 2024-05-27T06:56:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:48.858+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f544002e5e0 con 0x7f5440110bf0 2024-05-27T06:56:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.199+0000 7f5436a00700 1 -- 172.21.15.16:0/751992115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v30) ==== 115+0+0 (secure 0 0 0) 0x7f54380aa130 con 0x7f5440110bf0 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.204+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 msgr2=0x7f54240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.204+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 0x7f54240b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f543000b540 comp rx=0 tx=0).stop 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.204+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 msgr2=0x7f54401a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.204+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54380094f0 tx=0x7f5438007f90 comp rx=0 tx=0).stop 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 shutdown_connections 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f54240b2900 0x7f54240b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5440110bf0 0x7f54401a17b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5440075af0 0x7f544019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 --2- 172.21.15.16:0/751992115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440076440 0x7f544019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 >> 172.21.15.16:0/751992115 conn(0x7f5440100030 msgr2=0x7f5440100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.205+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 shutdown_connections 2024-05-27T06:56:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:49.206+0000 7f5445000700 1 -- 172.21.15.16:0/751992115 wait complete. 2024-05-27T06:56:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:49 smithi164 ceph-mon[35327]: pgmap v445: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.4 KiB/s wr, 24 op/s 2024-05-27T06:56:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:49 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T06:56:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:49 smithi016 ceph-mon[31415]: pgmap v445: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.4 KiB/s wr, 24 op/s 2024-05-27T06:56:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:49 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T06:56:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:49 smithi069 ceph-mon[35276]: pgmap v445: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.4 KiB/s wr, 24 op/s 2024-05-27T06:56:49.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:49 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T06:56:50.231 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-27T06:56:50.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-27T06:56:50.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:50 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T06:56:50.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:50 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:50 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T06:56:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:50 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:50.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:50 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/751992115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T06:56:50.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:50 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T06:56:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:51.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:51 smithi164 ceph-mon[35327]: pgmap v446: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-05-27T06:56:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:51 smithi016 ceph-mon[31415]: pgmap v446: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-05-27T06:56:51.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:51 smithi069 ceph-mon[35276]: pgmap v446: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 -- 172.21.15.16:0/371216800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106570 msgr2=0x7fe944106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 --2- 172.21.15.16:0/371216800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106570 0x7fe944106980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe930009990 tx=0x7fe930009ca0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 -- 172.21.15.16:0/371216800 shutdown_connections 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 --2- 172.21.15.16:0/371216800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe944106ec0 0x7fe944113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 --2- 172.21.15.16:0/371216800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94410c5b0 0x7fe94410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 --2- 172.21.15.16:0/371216800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106570 0x7fe944106980 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 -- 172.21.15.16:0/371216800 >> 172.21.15.16:0/371216800 conn(0x7fe944102070 msgr2=0x7fe9441044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:53.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 -- 172.21.15.16:0/371216800 shutdown_connections 2024-05-27T06:56:53.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.142+0000 7fe948e00700 1 -- 172.21.15.16:0/371216800 wait complete. 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.143+0000 7fe948e00700 1 Processor -- start 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.143+0000 7fe948e00700 1 -- start start 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe944106570 0x7fe94419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe94410c5b0 0x7fe9441a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe944116590 con 0x7fe944106ec0 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe944116410 con 0x7fe94410c5b0 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe948e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe944116710 con 0x7fe944106570 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe941400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe941400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33484/0 (socket says 172.21.15.16:33484) 2024-05-27T06:56:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe941400700 1 -- 172.21.15.16:0/4177585900 learned_addr learned my addr 172.21.15.16:0/4177585900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.144+0000 7fe941e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe944106570 0x7fe94419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe942800700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe94410c5b0 0x7fe9441a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 -- 172.21.15.16:0/4177585900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe944106570 msgr2=0x7fe94419d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe944106570 0x7fe94419d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 -- 172.21.15.16:0/4177585900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe94410c5b0 msgr2=0x7fe9441a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe94410c5b0 0x7fe9441a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 -- 172.21.15.16:0/4177585900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441a1f10 con 0x7fe944106ec0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.145+0000 7fe941400700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098e0 tx=0x7fe9300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.146+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930026070 con 0x7fe944106ec0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.146+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe93000fb00 con 0x7fe944106ec0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.146+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930021bd0 con 0x7fe944106ec0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.146+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441a21a0 con 0x7fe944106ec0 2024-05-27T06:56:53.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.146+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441a2630 con 0x7fe944106ec0 2024-05-27T06:56:53.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.147+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9440512b0 con 0x7fe944106ec0 2024-05-27T06:56:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.150+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe930029070 con 0x7fe944106ec0 2024-05-27T06:56:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.151+0000 7fe93aa00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 0x7fe9280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.151+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe9300e34d0 con 0x7fe944106ec0 2024-05-27T06:56:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.152+0000 7fe941e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 0x7fe9280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:53.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.152+0000 7fe941e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 0x7fe9280b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe934005950 tx=0x7fe93400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:53.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.154+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe93005edf0 con 0x7fe944106ec0 2024-05-27T06:56:53.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:53 smithi016 ceph-mon[31415]: pgmap v447: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T06:56:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.434+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe944063bf0 con 0x7fe9280b2900 2024-05-27T06:56:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.529+0000 7fe93aa00700 1 -- 172.21.15.16:0/4177585900 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe944063bf0 con 0x7fe9280b2900 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 msgr2=0x7fe9280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 0x7fe9280b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe934005950 tx=0x7fe93400b410 comp rx=0 tx=0).stop 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 msgr2=0x7fe94419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098e0 tx=0x7fe9300057c0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 shutdown_connections 2024-05-27T06:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9280b2900 0x7fe9280b4e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe94410c5b0 0x7fe9441a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe944106570 0x7fe94419d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 --2- 172.21.15.16:0/4177585900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944106ec0 0x7fe94419d5e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 >> 172.21.15.16:0/4177585900 conn(0x7fe944102070 msgr2=0x7fe9441049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.533+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 shutdown_connections 2024-05-27T06:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:53.534+0000 7fe948e00700 1 -- 172.21.15.16:0/4177585900 wait complete. 2024-05-27T06:56:53.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:53 smithi164 ceph-mon[35327]: pgmap v447: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T06:56:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:53 smithi069 ceph-mon[35276]: pgmap v447: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T06:56:53.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:56:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:56:53.448+0000 7fde21c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T06:56:53.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:56:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:56:53.448+0000 7fde21c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T06:56:53.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:56:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:56:53.448+0000 7fde21c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T06:56:53.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:56:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:56:53.448+0000 7fde21c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T06:56:53.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 06:56:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T06:56:53.448+0000 7fde21c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T06:56:54.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-27T06:56:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:55.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:55 smithi016 ceph-mon[31415]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:55.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:55 smithi016 ceph-mon[31415]: pgmap v448: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-27T06:56:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:55 smithi164 ceph-mon[35327]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:55.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:55 smithi164 ceph-mon[35327]: pgmap v448: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-27T06:56:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:55 smithi069 ceph-mon[35276]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:55 smithi069 ceph-mon[35276]: pgmap v448: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-27T06:56:56.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:56 smithi016 ceph-mon[31415]: mgrmap e17: x(active, since 12m), standbys: z, y 2024-05-27T06:56:56.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:56 smithi164 ceph-mon[35327]: mgrmap e17: x(active, since 12m), standbys: z, y 2024-05-27T06:56:56.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:56 smithi069 ceph-mon[35276]: mgrmap e17: x(active, since 12m), standbys: z, y 2024-05-27T06:56:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 -- 172.21.15.16:0/1860367021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108108f10 msgr2=0x7fd108109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 --2- 172.21.15.16:0/1860367021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108108f10 0x7fd108109320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40099c0 tx=0x7fd0f4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 -- 172.21.15.16:0/1860367021 shutdown_connections 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 --2- 172.21.15.16:0/1860367021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108109860 0x7fd1081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 --2- 172.21.15.16:0/1860367021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081045e0 0x7fd1081049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 --2- 172.21.15.16:0/1860367021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108108f10 0x7fd108109320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 -- 172.21.15.16:0/1860367021 >> 172.21.15.16:0/1860367021 conn(0x7fd108100030 msgr2=0x7fd108102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 -- 172.21.15.16:0/1860367021 shutdown_connections 2024-05-27T06:56:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.374+0000 7fd10d600700 1 -- 172.21.15.16:0/1860367021 wait complete. 2024-05-27T06:56:57.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:57 smithi016 ceph-mon[31415]: pgmap v449: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.375+0000 7fd10d600700 1 Processor -- start 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 -- start start 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108108f10 0x7fd10819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd108109860 0x7fd10819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081143e0 con 0x7fd1081045e0 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd108114260 con 0x7fd108109860 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.376+0000 7fd10d600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd108114560 con 0x7fd108108f10 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33518/0 (socket says 172.21.15.16:33518) 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 -- 172.21.15.16:0/3849382706 learned_addr learned my addr 172.21.15.16:0/3849382706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106000700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108108f10 0x7fd10819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd107400700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd108109860 0x7fd10819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 -- 172.21.15.16:0/3849382706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108108f10 msgr2=0x7fd10819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108108f10 0x7fd10819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 -- 172.21.15.16:0/3849382706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd108109860 msgr2=0x7fd10819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd108109860 0x7fd10819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.377+0000 7fd106a00700 1 -- 172.21.15.16:0/3849382706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10819fd00 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd106a00700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800d950 tx=0x7fd0f800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8016070 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f8010460 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800f620 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10819fff0 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.378+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081a7910 con 0x7fd1081045e0 2024-05-27T06:56:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.380+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd108105a90 con 0x7fd1081045e0 2024-05-27T06:56:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.382+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd0f8019070 con 0x7fd1081045e0 2024-05-27T06:56:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.384+0000 7fd0ff400700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 0x7fd0ec0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:56:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.385+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f80e2340 con 0x7fd1081045e0 2024-05-27T06:56:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.385+0000 7fd106000700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 0x7fd0ec0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:56:57.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.386+0000 7fd106000700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 0x7fd0ec0b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009990 tx=0x7fd0f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:56:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.390+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0f80a9af0 con 0x7fd1081045e0 2024-05-27T06:56:57.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:57 smithi164 ceph-mon[35327]: pgmap v449: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-27T06:56:57.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:57 smithi069 ceph-mon[35276]: pgmap v449: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-27T06:56:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.662+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd108063bf0 con 0x7fd0ec0b2990 2024-05-27T06:56:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.741+0000 7fd0ff400700 1 -- 172.21.15.16:0/3849382706 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd108063bf0 con 0x7fd0ec0b2990 2024-05-27T06:56:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 msgr2=0x7fd0ec0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 0x7fd0ec0b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009990 tx=0x7fd0f4005fb0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 msgr2=0x7fd10819ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800d950 tx=0x7fd0f800dd10 comp rx=0 tx=0).stop 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 shutdown_connections 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0ec0b2990 0x7fd0ec0b4ea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd108109860 0x7fd10819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1081045e0 0x7fd10819ae30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 --2- 172.21.15.16:0/3849382706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd108108f10 0x7fd10819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.745+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 >> 172.21.15.16:0/3849382706 conn(0x7fd108100030 msgr2=0x7fd10810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.746+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 shutdown_connections 2024-05-27T06:56:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:56:57.746+0000 7fd10d600700 1 -- 172.21.15.16:0/3849382706 wait complete. 2024-05-27T06:56:58.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-27T06:56:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:56:59.561 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:59 smithi016 ceph-mon[31415]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:59.561 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:56:59 smithi016 ceph-mon[31415]: pgmap v450: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T06:56:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:59 smithi164 ceph-mon[35327]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:56:59 smithi164 ceph-mon[35327]: pgmap v450: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T06:56:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:59 smithi069 ceph-mon[35276]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:56:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:56:59 smithi069 ceph-mon[35276]: pgmap v450: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T06:57:01.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:01 smithi016 ceph-mon[31415]: pgmap v451: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T06:57:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:01 smithi164 ceph-mon[35327]: pgmap v451: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.622+0000 7fdaac000700 1 -- 172.21.15.16:0/567988393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4105e20 msgr2=0x7fdaa410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.622+0000 7fdaac000700 1 --2- 172.21.15.16:0/567988393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4105e20 0x7fdaa410e370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda90009990 tx=0x7fda90009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 -- 172.21.15.16:0/567988393 shutdown_connections 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 --2- 172.21.15.16:0/567988393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa410e900 0x7fdaa4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 --2- 172.21.15.16:0/567988393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa41054d0 0x7fdaa41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 --2- 172.21.15.16:0/567988393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4105e20 0x7fdaa410e370 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 -- 172.21.15.16:0/567988393 >> 172.21.15.16:0/567988393 conn(0x7fdaa40fedb0 msgr2=0x7fdaa41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 -- 172.21.15.16:0/567988393 shutdown_connections 2024-05-27T06:57:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.623+0000 7fdaac000700 1 -- 172.21.15.16:0/567988393 wait complete. 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.624+0000 7fdaac000700 1 Processor -- start 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.624+0000 7fdaac000700 1 -- start start 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa4105e20 0x7fdaa419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 0x7fdaa41a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa41142e0 con 0x7fdaa41054d0 2024-05-27T06:57:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4114160 con 0x7fdaa410e900 2024-05-27T06:57:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaac000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4114460 con 0x7fdaa4105e20 2024-05-27T06:57:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaa9600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaa9600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33544/0 (socket says 172.21.15.16:33544) 2024-05-27T06:57:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.625+0000 7fdaa9600700 1 -- 172.21.15.16:0/300112849 learned_addr learned my addr 172.21.15.16:0/300112849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:01.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:01 smithi069 ceph-mon[35276]: pgmap v451: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa8c00700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa4105e20 0x7fdaa419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaaa000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 0x7fdaa41a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 -- 172.21.15.16:0/300112849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa4105e20 msgr2=0x7fdaa419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa4105e20 0x7fdaa419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 -- 172.21.15.16:0/300112849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 msgr2=0x7fdaa41a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 0x7fdaa41a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 -- 172.21.15.16:0/300112849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa41a1de0 con 0x7fdaa41054d0 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaaa000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 0x7fdaa41a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:57:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.626+0000 7fdaa9600700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d950 tx=0x7fda9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.627+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98016070 con 0x7fdaa41054d0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.627+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda98009dd0 con 0x7fdaa41054d0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.627+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa41a20d0 con 0x7fdaa41054d0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.627+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9800f3c0 con 0x7fdaa41054d0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.627+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41a99f0 con 0x7fdaa41054d0 2024-05-27T06:57:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.628+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa40512b0 con 0x7fdaa41054d0 2024-05-27T06:57:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.633+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fda98019070 con 0x7fdaa41054d0 2024-05-27T06:57:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.634+0000 7fdaa2000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 0x7fda8c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.634+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fda980e2410 con 0x7fdaa41054d0 2024-05-27T06:57:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.634+0000 7fdaa8c00700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 0x7fda8c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.635+0000 7fdaa8c00700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 0x7fda8c0b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b5c0 tx=0x7fda90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.641+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda980a9bc0 con 0x7fdaa41054d0 2024-05-27T06:57:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.916+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdaa4063bf0 con 0x7fda8c0b2910 2024-05-27T06:57:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.926+0000 7fdaa2000700 1 -- 172.21.15.16:0/300112849 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdaa4063bf0 con 0x7fda8c0b2910 2024-05-27T06:57:01.919 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/c2880afe-8509-41f8-835f-a35eb3c9cbaf 2024-05-27T06:57:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 msgr2=0x7fda8c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 0x7fda8c0b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b5c0 tx=0x7fda90005960 comp rx=0 tx=0).stop 2024-05-27T06:57:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 msgr2=0x7fdaa419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d950 tx=0x7fda9800dc60 comp rx=0 tx=0).stop 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 shutdown_connections 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fda8c0b2910 0x7fda8c0b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa410e900 0x7fdaa41a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa41054d0 0x7fdaa419cf10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 --2- 172.21.15.16:0/300112849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa4105e20 0x7fdaa419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 >> 172.21.15.16:0/300112849 conn(0x7fdaa40fedb0 msgr2=0x7fdaa40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 shutdown_connections 2024-05-27T06:57:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:01.930+0000 7fdaac000700 1 -- 172.21.15.16:0/300112849 wait complete. 2024-05-27T06:57:02.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-27T06:57:03.315 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:03 smithi164 ceph-mon[35327]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:03.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:03 smithi164 ceph-mon[35327]: pgmap v452: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-27T06:57:03.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:03 smithi016 ceph-mon[31415]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:03.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:03 smithi016 ceph-mon[31415]: pgmap v452: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-27T06:57:03.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:03 smithi069 ceph-mon[35276]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:03.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:03 smithi069 ceph-mon[35276]: pgmap v452: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-27T06:57:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:05 smithi016 ceph-mon[31415]: pgmap v453: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:05.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:05 smithi164 ceph-mon[35327]: pgmap v453: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:05.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:05 smithi069 ceph-mon[35276]: pgmap v453: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 -- 172.21.15.16:0/2235594840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c075d20 msgr2=0x7f0d0c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2235594840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c075d20 0x7f0d0c076130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009990 tx=0x7f0d00009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 -- 172.21.15.16:0/2235594840 shutdown_connections 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2235594840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 0x7f0d0c112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2235594840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c075d20 0x7f0d0c076130 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2235594840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c076670 0x7f0d0c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 -- 172.21.15.16:0/2235594840 >> 172.21.15.16:0/2235594840 conn(0x7f0d0c100030 msgr2=0x7f0d0c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 -- 172.21.15.16:0/2235594840 shutdown_connections 2024-05-27T06:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.935+0000 7f0d13a00700 1 -- 172.21.15.16:0/2235594840 wait complete. 2024-05-27T06:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 Processor -- start 2024-05-27T06:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 -- start start 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c075d20 0x7f0d0c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 0x7f0d0c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c116580 con 0x7f0d0c076670 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c116400 con 0x7f0d0c110b90 2024-05-27T06:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.937+0000 7f0d13a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c116700 con 0x7f0d0c075d20 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51686/0 (socket says 172.21.15.16:51686) 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 -- 172.21.15.16:0/2381425783 learned_addr learned my addr 172.21.15.16:0/2381425783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d11000700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c075d20 0x7f0d0c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d11a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 0x7f0d0c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 -- 172.21.15.16:0/2381425783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c075d20 msgr2=0x7f0d0c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c075d20 0x7f0d0c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 -- 172.21.15.16:0/2381425783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 msgr2=0x7f0d0c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 0x7f0d0c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.938+0000 7f0d0be00700 1 -- 172.21.15.16:0/2381425783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c1a1e20 con 0x7f0d0c076670 2024-05-27T06:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d0be00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d950 tx=0x7f0cf800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8016070 con 0x7f0d0c076670 2024-05-27T06:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cf8010460 con 0x7f0d0c076670 2024-05-27T06:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f620 con 0x7f0d0c076670 2024-05-27T06:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1a2110 con 0x7f0d0c076670 2024-05-27T06:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.939+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a9a30 con 0x7f0d0c076670 2024-05-27T06:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.940+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0c0512b0 con 0x7f0d0c076670 2024-05-27T06:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.945+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0cf8019070 con 0x7f0d0c076670 2024-05-27T06:57:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.946+0000 7f0d09600700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 0x7f0cf40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.946+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0cf80e2360 con 0x7f0d0c076670 2024-05-27T06:57:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.946+0000 7f0d11000700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 0x7f0cf40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.947+0000 7f0d11000700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 0x7f0cf40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000b5c0 tx=0x7f0d00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:05.949+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cf80a9b70 con 0x7f0d0c076670 2024-05-27T06:57:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.225+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0d0c19e2b0 con 0x7f0cf40b2910 2024-05-27T06:57:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.312+0000 7f0d09600700 1 -- 172.21.15.16:0/2381425783 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d0c19e2b0 con 0x7f0cf40b2910 2024-05-27T06:57:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.315+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 msgr2=0x7f0cf40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.315+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 0x7f0cf40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000b5c0 tx=0x7f0d00005fb0 comp rx=0 tx=0).stop 2024-05-27T06:57:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.315+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 msgr2=0x7f0d0c19d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.315+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d950 tx=0x7f0cf800dd10 comp rx=0 tx=0).stop 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 shutdown_connections 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0cf40b2910 0x7f0cf40b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d0c110b90 0x7f0d0c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c075d20 0x7f0d0c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 --2- 172.21.15.16:0/2381425783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d0c076670 0x7f0d0c19d4f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 >> 172.21.15.16:0/2381425783 conn(0x7f0d0c100030 msgr2=0x7f0d0c100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 shutdown_connections 2024-05-27T06:57:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:06.316+0000 7f0d13a00700 1 -- 172.21.15.16:0/2381425783 wait complete. 2024-05-27T06:57:07.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-27T06:57:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:07.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:07 smithi016 ceph-mon[31415]: pgmap v454: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T06:57:07.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:07 smithi016 ceph-mon[31415]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:07 smithi069 ceph-mon[35276]: pgmap v454: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T06:57:07.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:07 smithi069 ceph-mon[35276]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:07 smithi164 ceph-mon[35327]: pgmap v454: 129 pgs: 129 active+clean; 614 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T06:57:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:07 smithi164 ceph-mon[35327]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:09.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:09 smithi016 ceph-mon[31415]: pgmap v455: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T06:57:09.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:09 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:57:09.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:09 smithi069 ceph-mon[35276]: pgmap v455: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T06:57:09.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:09 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:57:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:09 smithi164 ceph-mon[35327]: pgmap v455: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T06:57:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:09 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:57:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 -- 172.21.15.16:0/1960092499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e01045e0 msgr2=0x7f88e01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 --2- 172.21.15.16:0/1960092499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e01045e0 0x7f88e01049f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f88d40099c0 tx=0x7f88d4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 -- 172.21.15.16:0/1960092499 shutdown_connections 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 --2- 172.21.15.16:0/1960092499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e0109860 0x7f88e01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 --2- 172.21.15.16:0/1960092499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e01045e0 0x7f88e01049f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 --2- 172.21.15.16:0/1960092499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 0x7f88e0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 -- 172.21.15.16:0/1960092499 >> 172.21.15.16:0/1960092499 conn(0x7f88e0100030 msgr2=0x7f88e0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 -- 172.21.15.16:0/1960092499 shutdown_connections 2024-05-27T06:57:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.207+0000 7f88e8800700 1 -- 172.21.15.16:0/1960092499 wait complete. 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 Processor -- start 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 -- start start 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 0x7f88e019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e0109860 0x7f88e01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0114390 con 0x7f88e0109860 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0114210 con 0x7f88e01045e0 2024-05-27T06:57:10.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.209+0000 7f88e8800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0114510 con 0x7f88e0108f10 2024-05-27T06:57:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 0x7f88e019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51112/0 (socket says 172.21.15.16:51112) 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 -- 172.21.15.16:0/2804192182 learned_addr learned my addr 172.21.15.16:0/2804192182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e6800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e0109860 0x7f88e01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 -- 172.21.15.16:0/2804192182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 msgr2=0x7f88e019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 0x7f88e019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 -- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e0109860 msgr2=0x7f88e01a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e0109860 0x7f88e01a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.210+0000 7f88e5e00700 1 -- 172.21.15.16:0/2804192182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e01a1ef0 con 0x7f88e01045e0 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88e5e00700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006010 tx=0x7f88d4005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4026070 con 0x7f88e01045e0 2024-05-27T06:57:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e01a2180 con 0x7f88e01045e0 2024-05-27T06:57:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88d4005680 con 0x7f88e01045e0 2024-05-27T06:57:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d400faa0 con 0x7f88e01045e0 2024-05-27T06:57:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.211+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e01a9b00 con 0x7f88e01045e0 2024-05-27T06:57:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.212+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e00512b0 con 0x7f88e01045e0 2024-05-27T06:57:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.215+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f88d4029070 con 0x7f88e01045e0 2024-05-27T06:57:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.218+0000 7f88daa00700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 0x7f88c80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.218+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f88d40e36e0 con 0x7f88e01045e0 2024-05-27T06:57:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.219+0000 7f88e5400700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 0x7f88c80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.220+0000 7f88e5400700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 0x7f88c80b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88e0107e20 tx=0x7f88d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.220+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88d4030080 con 0x7f88e01045e0 2024-05-27T06:57:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.516+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f88e00769f0 con 0x7f88c80b2990 2024-05-27T06:57:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.528+0000 7f88daa00700 1 -- 172.21.15.16:0/2804192182 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f88e00769f0 con 0x7f88c80b2990 2024-05-27T06:57:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/b2debf6f-5b83-4a5e-8695-fd25ab1566b3 2024-05-27T06:57:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.531+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 msgr2=0x7f88c80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.531+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 0x7f88c80b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88e0107e20 tx=0x7f88d0009260 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.531+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 msgr2=0x7f88e019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.531+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006010 tx=0x7f88d4005280 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 shutdown_connections 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88c80b2990 0x7f88c80b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88e0109860 0x7f88e01a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e01045e0 0x7f88e019d020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 --2- 172.21.15.16:0/2804192182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e0108f10 0x7f88e019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 >> 172.21.15.16:0/2804192182 conn(0x7f88e0100030 msgr2=0x7f88e010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 shutdown_connections 2024-05-27T06:57:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:10.532+0000 7f88e8800700 1 -- 172.21.15.16:0/2804192182 wait complete. 2024-05-27T06:57:11.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:11 smithi016 ceph-mon[31415]: pgmap v456: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-27T06:57:11.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:11 smithi069 ceph-mon[35276]: pgmap v456: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-27T06:57:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:11 smithi164 ceph-mon[35327]: pgmap v456: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-27T06:57:12.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:12.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:12 smithi016 ceph-mon[31415]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:12.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:12 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:12 smithi069 ceph-mon[35276]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:12 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:12 smithi164 ceph-mon[35327]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:57:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:12 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:13.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:13 smithi016 ceph-mon[31415]: pgmap v457: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-27T06:57:13.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:13 smithi069 ceph-mon[35276]: pgmap v457: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-27T06:57:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:13 smithi164 ceph-mon[35327]: pgmap v457: 129 pgs: 129 active+clean; 631 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-27T06:57:14.711 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.711 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:14 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.711 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:14 smithi016 ceph-mon[31415]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-27T06:57:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:14 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:14 smithi164 ceph-mon[35327]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-27T06:57:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:14 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:14.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:14 smithi069 ceph-mon[35276]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-27T06:57:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 -- 172.21.15.16:0/2603874984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 msgr2=0x7fdfb010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/2603874984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb010e3b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009990 tx=0x7fdf9c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 -- 172.21.15.16:0/2603874984 shutdown_connections 2024-05-27T06:57:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/2603874984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb0110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/2603874984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/2603874984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb010e3b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 -- 172.21.15.16:0/2603874984 >> 172.21.15.16:0/2603874984 conn(0x7fdfb00fedb0 msgr2=0x7fdfb01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 -- 172.21.15.16:0/2603874984 shutdown_connections 2024-05-27T06:57:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.966+0000 7fdfb4a00700 1 -- 172.21.15.16:0/2603874984 wait complete. 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.967+0000 7fdfb4a00700 1 Processor -- start 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.967+0000 7fdfb4a00700 1 -- start start 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01142e0 con 0x7fdfb0105e60 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0114160 con 0x7fdfb0105510 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfb4a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0114460 con 0x7fdfb010e8f0 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfad400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfad400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41450/0 (socket says 172.21.15.16:41450) 2024-05-27T06:57:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.968+0000 7fdfad400700 1 -- 172.21.15.16:0/4239328196 learned_addr learned my addr 172.21.15.16:0/4239328196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfade00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfae800700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 -- 172.21.15.16:0/4239328196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 msgr2=0x7fdfb01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 -- 172.21.15.16:0/4239328196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 msgr2=0x7fdfb019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 -- 172.21.15.16:0/4239328196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb01a1d50 con 0x7fdfb0105e60 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfae800700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfade00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.969+0000 7fdfad400700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c000c00 tx=0x7fdf9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.970+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c027070 con 0x7fdfb0105e60 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.970+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf9c00fc70 con 0x7fdfb0105e60 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.970+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb01a1fe0 con 0x7fdfb0105e60 2024-05-27T06:57:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.970+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01a24c0 con 0x7fdfb0105e60 2024-05-27T06:57:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.970+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c022d40 con 0x7fdfb0105e60 2024-05-27T06:57:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.972+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb00512b0 con 0x7fdfb0105e60 2024-05-27T06:57:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.974+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdf9c02a070 con 0x7fdfb0105e60 2024-05-27T06:57:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.977+0000 7fdfa6a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 0x7fdf940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.977+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdf9c0e35b0 con 0x7fdfb0105e60 2024-05-27T06:57:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.977+0000 7fdfade00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 0x7fdf940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.978+0000 7fdfade00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 0x7fdf940b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005950 tx=0x7fdfa00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:14.982+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf9c013070 con 0x7fdfb0105e60 2024-05-27T06:57:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.275+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfb002e5e0 con 0x7fdfb0105e60 2024-05-27T06:57:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.276+0000 7fdfa6a00700 1 -- 172.21.15.16:0/4239328196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7fdf9c00fe20 con 0x7fdfb0105e60 2024-05-27T06:57:15.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 msgr2=0x7fdf940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 0x7fdf940b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005950 tx=0x7fdfa00058e0 comp rx=0 tx=0).stop 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 msgr2=0x7fdfb019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c000c00 tx=0x7fdf9c009f90 comp rx=0 tx=0).stop 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 shutdown_connections 2024-05-27T06:57:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf940b2910 0x7fdf940b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfb010e8f0 0x7fdfb01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfb0105510 0x7fdfb019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 --2- 172.21.15.16:0/4239328196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0105e60 0x7fdfb019d4d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.280+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 >> 172.21.15.16:0/4239328196 conn(0x7fdfb00fedb0 msgr2=0x7fdfb0103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.281+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 shutdown_connections 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:15.281+0000 7fdfb4a00700 1 -- 172.21.15.16:0/4239328196 wait complete. 2024-05-27T06:57:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T06:57:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:57:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:57:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:15.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:15 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4239328196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:57:15.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:57:15.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:57:15.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:15 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4239328196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:57:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:57:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:57:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:57:15.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:15 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4239328196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:57:16.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:57:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:16 smithi164 ceph-mon[35327]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-27T06:57:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:16 smithi016 ceph-mon[31415]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-27T06:57:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:16 smithi069 ceph-mon[35276]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-27T06:57:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 -- 172.21.15.16:0/2640013988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7810af40 msgr2=0x7fce7810b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 --2- 172.21.15.16:0/2640013988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7810af40 0x7fce7810b350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fce6000b0d0 tx=0x7fce6000b3e0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 -- 172.21.15.16:0/2640013988 shutdown_connections 2024-05-27T06:57:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 --2- 172.21.15.16:0/2640013988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce7810b890 0x7fce781131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 --2- 172.21.15.16:0/2640013988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce78106610 0x7fce78106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 --2- 172.21.15.16:0/2640013988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7810af40 0x7fce7810b350 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 -- 172.21.15.16:0/2640013988 >> 172.21.15.16:0/2640013988 conn(0x7fce78102020 msgr2=0x7fce78104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.076+0000 7fce7da00700 1 -- 172.21.15.16:0/2640013988 shutdown_connections 2024-05-27T06:57:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.077+0000 7fce7da00700 1 -- 172.21.15.16:0/2640013988 wait complete. 2024-05-27T06:57:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.078+0000 7fce7da00700 1 Processor -- start 2024-05-27T06:57:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.078+0000 7fce7da00700 1 -- start start 2024-05-27T06:57:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7810b890 0x7fce7819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce781a1670 0x7fce781a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce78116350 con 0x7fce78106610 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce781161d0 con 0x7fce781a1670 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce781164d0 con 0x7fce7810b890 2024-05-27T06:57:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce77e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce77e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41462/0 (socket says 172.21.15.16:41462) 2024-05-27T06:57:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce77e00700 1 -- 172.21.15.16:0/830803447 learned_addr learned my addr 172.21.15.16:0/830803447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce6f400700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7810b890 0x7fce7819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.079+0000 7fce7d000700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce781a1670 0x7fce781a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 -- 172.21.15.16:0/830803447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7810b890 msgr2=0x7fce7819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7810b890 0x7fce7819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 -- 172.21.15.16:0/830803447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce781a1670 msgr2=0x7fce781a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce781a1670 0x7fce781a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 -- 172.21.15.16:0/830803447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce781a21c0 con 0x7fce78106610 2024-05-27T06:57:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.080+0000 7fce77e00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c920 tx=0x7fce6800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.081+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004a30 con 0x7fce78106610 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.081+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce781a24b0 con 0x7fce78106610 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.081+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce680054b0 con 0x7fce78106610 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.081+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68007b70 con 0x7fce78106610 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.081+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce781a9c60 con 0x7fce78106610 2024-05-27T06:57:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.082+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce780512b0 con 0x7fce78106610 2024-05-27T06:57:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.087+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fce68004bd0 con 0x7fce78106610 2024-05-27T06:57:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.088+0000 7fce75600700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 0x7fce5c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.088+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fce680e2680 con 0x7fce78106610 2024-05-27T06:57:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.088+0000 7fce6f400700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 0x7fce5c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.089+0000 7fce6f400700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 0x7fce5c0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce60009040 tx=0x7fce6000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.091+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce68014030 con 0x7fce78106610 2024-05-27T06:57:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.323+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce7802e5e0 con 0x7fce78106610 2024-05-27T06:57:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.328+0000 7fce75600700 1 -- 172.21.15.16:0/830803447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24708 (secure 0 0 0) 0x7fce68005660 con 0x7fce78106610 2024-05-27T06:57:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:19.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:19 smithi016 ceph-mon[31415]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:19.321 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:56:40.328992+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:57:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 msgr2=0x7fce5c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 0x7fce5c0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce60009040 tx=0x7fce6000b040 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 msgr2=0x7fce7819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c920 tx=0x7fce6800cc30 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 shutdown_connections 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fce5c0b2910 0x7fce5c0b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce781a1670 0x7fce781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce78106610 0x7fce7819cf00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 --2- 172.21.15.16:0/830803447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7810b890 0x7fce7819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.331+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 >> 172.21.15.16:0/830803447 conn(0x7fce78102020 msgr2=0x7fce78104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.332+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 shutdown_connections 2024-05-27T06:57:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:19.332+0000 7fce7da00700 1 -- 172.21.15.16:0/830803447 wait complete. 2024-05-27T06:57:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:19 smithi164 ceph-mon[35327]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:19.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:19 smithi069 ceph-mon[35276]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-05-27T06:57:20.364 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T06:57:20.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T06:57:20.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:20 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/830803447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:20 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/830803447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:20.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:20 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/830803447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:21.330 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:21 smithi016 ceph-mon[31415]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:21 smithi164 ceph-mon[35327]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:21 smithi069 ceph-mon[35276]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 -- 172.21.15.16:0/1676729822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 msgr2=0x7f5eb8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 --2- 172.21.15.16:0/1676729822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb8106970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 -- 172.21.15.16:0/1676729822 shutdown_connections 2024-05-27T06:57:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 --2- 172.21.15.16:0/1676729822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb8106eb0 0x7f5eb8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 --2- 172.21.15.16:0/1676729822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb810c5a0 0x7f5eb810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 --2- 172.21.15.16:0/1676729822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb8106970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 -- 172.21.15.16:0/1676729822 >> 172.21.15.16:0/1676729822 conn(0x7f5eb8102040 msgr2=0x7f5eb8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 -- 172.21.15.16:0/1676729822 shutdown_connections 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.171+0000 7f5ec0800700 1 -- 172.21.15.16:0/1676729822 wait complete. 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 Processor -- start 2024-05-27T06:57:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 -- start start 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 0x7f5eb819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb810c5a0 0x7f5eb81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8116580 con 0x7f5eb8106560 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8116400 con 0x7f5eb810c5a0 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.173+0000 7f5ec0800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8116700 con 0x7f5eb8106eb0 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55252/0 (socket says 172.21.15.16:55252) 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 -- 172.21.15.16:0/3453806768 learned_addr learned my addr 172.21.15.16:0/3453806768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebd400700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 0x7f5eb819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 -- 172.21.15.16:0/3453806768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 msgr2=0x7f5eb819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 0x7f5eb819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 -- 172.21.15.16:0/3453806768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb810c5a0 msgr2=0x7f5eb81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb810c5a0 0x7f5eb81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebde00700 1 -- 172.21.15.16:0/3453806768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb81a1e00 con 0x7f5eb8106560 2024-05-27T06:57:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.174+0000 7f5ebd400700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 0x7f5eb819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5ebde00700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00e970 tx=0x7f5eac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00cbd0 con 0x7f5eb8106560 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5eac00cd70 con 0x7f5eb8106560 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac005610 con 0x7f5eb8106560 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81a20f0 con 0x7f5eb8106560 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.175+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81a9a10 con 0x7f5eb8106560 2024-05-27T06:57:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.177+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb8107c80 con 0x7f5eb8106560 2024-05-27T06:57:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.180+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5eac004750 con 0x7f5eb8106560 2024-05-27T06:57:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.181+0000 7f5eb2a00700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 0x7f5ea00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.182+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5eac0e2c80 con 0x7f5eb8106560 2024-05-27T06:57:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.182+0000 7f5ebd400700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 0x7f5ea00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.183+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5eac054df0 con 0x7f5eb8106560 2024-05-27T06:57:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.183+0000 7f5ebd400700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 0x7f5ea00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800b650 tx=0x7f5ea8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:23.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.419+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5eb819e2f0 con 0x7f5ea00b2910 2024-05-27T06:57:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.433+0000 7f5eb2a00700 1 -- 172.21.15.16:0/3453806768 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340455 (secure 0 0 0) 0x7f5eb819e2f0 con 0x7f5ea00b2910 2024-05-27T06:57:23.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:23 smithi016 ceph-mon[31415]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:23.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 msgr2=0x7f5ea00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 0x7f5ea00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800b650 tx=0x7f5ea8005fb0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 msgr2=0x7f5eb819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00e970 tx=0x7f5eac00ed30 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 shutdown_connections 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5ea00b2910 0x7f5ea00b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb810c5a0 0x7f5eb81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb8106560 0x7f5eb819cf30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 --2- 172.21.15.16:0/3453806768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8106eb0 0x7f5eb819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.440+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 >> 172.21.15.16:0/3453806768 conn(0x7f5eb8102040 msgr2=0x7f5eb81038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:23.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.441+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 shutdown_connections 2024-05-27T06:57:23.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:23.441+0000 7f5ec0800700 1 -- 172.21.15.16:0/3453806768 wait complete. 2024-05-27T06:57:23.438 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T06:57:23.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:23 smithi164 ceph-mon[35327]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:23.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:23 smithi069 ceph-mon[35276]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-27T06:57:24.451 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-05-27T06:57:22.159736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660413,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":477,"num_read_kb":339,"num_write":376,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380188,"kb_used_data":14392,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561636,"statfs":{"total":1151940427776,"available":1151551115264,"internally_reserved":0,"allocated":14737408,"data_stored":8211207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13842,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":69,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006593"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408775+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T06:56:40.408775+0000","last_peered":"2024-05-27T06:56:40.408775+0000","last_clean":"2024-05-27T06:56:40.408775+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T06:56:40.408775+0000","last_undegraded":"2024-05-27T06:56:40.408775+0000","last_fullsized":"2024-05-27T06:56:40.408775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399796+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T06:56:40.399796+0000","last_peered":"2024-05-27T06:56:40.399796+0000","last_clean":"2024-05-27T06:56:40.399796+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T06:56:40.399796+0000","last_undegraded":"2024-05-27T06:56:40.399796+0000","last_fullsized":"2024-05-27T06:56:40.399796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399147+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T06:56:40.399147+0000","last_peered":"2024-05-27T06:56:40.399147+0000","last_clean":"2024-05-27T06:56:40.399147+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T06:56:40.399147+0000","last_undegraded":"2024-05-27T06:56:40.399147+0000","last_fullsized":"2024-05-27T06:56:40.399147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341588+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T06:56:40.341588+0000","last_peered":"2024-05-27T06:56:40.341588+0000","last_clean":"2024-05-27T06:56:40.341588+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T06:56:40.341588+0000","last_undegraded":"2024-05-27T06:56:40.341588+0000","last_fullsized":"2024-05-27T06:56:40.341588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401634+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T06:56:40.401634+0000","last_peered":"2024-05-27T06:56:40.401634+0000","last_clean":"2024-05-27T06:56:40.401634+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T06:56:40.401634+0000","last_undegraded":"2024-05-27T06:56:40.401634+0000","last_fullsized":"2024-05-27T06:56:40.401634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.392651+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T06:56:40.392651+0000","last_peered":"2024-05-27T06:56:40.392651+0000","last_clean":"2024-05-27T06:56:40.392651+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T06:56:40.392651+0000","last_undegraded":"2024-05-27T06:56:40.392651+0000","last_fullsized":"2024-05-27T06:56:40.392651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338271+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T06:56:40.338271+0000","last_peered":"2024-05-27T06:56:40.338271+0000","last_clean":"2024-05-27T06:56:40.338271+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T06:56:40.338271+0000","last_undegraded":"2024-05-27T06:56:40.338271+0000","last_fullsized":"2024-05-27T06:56:40.338271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404208+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T06:56:40.404208+0000","last_peered":"2024-05-27T06:56:40.404208+0000","last_clean":"2024-05-27T06:56:40.404208+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T06:56:40.404208+0000","last_undegraded":"2024-05-27T06:56:40.404208+0000","last_fullsized":"2024-05-27T06:56:40.404208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396944+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T06:56:40.396944+0000","last_peered":"2024-05-27T06:56:40.396944+0000","last_clean":"2024-05-27T06:56:40.396944+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T06:56:40.396944+0000","last_undegraded":"2024-05-27T06:56:40.396944+0000","last_fullsized":"2024-05-27T06:56:40.396944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394285+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T06:56:40.394285+0000","last_peered":"2024-05-27T06:56:40.394285+0000","last_clean":"2024-05-27T06:56:40.394285+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T06:56:40.394285+0000","last_undegraded":"2024-05-27T06:56:40.394285+0000","last_fullsized":"2024-05-27T06:56:40.394285+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339992+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T06:56:40.339992+0000","last_peered":"2024-05-27T06:56:40.339992+0000","last_clean":"2024-05-27T06:56:40.339992+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T06:56:40.339992+0000","last_undegraded":"2024-05-27T06:56:40.339992+0000","last_fullsized":"2024-05-27T06:56:40.339992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341754+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T06:56:40.341754+0000","last_peered":"2024-05-27T06:56:40.341754+0000","last_clean":"2024-05-27T06:56:40.341754+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T06:56:40.341754+0000","last_undegraded":"2024-05-27T06:56:40.341754+0000","last_fullsized":"2024-05-27T06:56:40.341754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335627+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T06:56:40.335627+0000","last_peered":"2024-05-27T06:56:40.335627+0000","last_clean":"2024-05-27T06:56:40.335627+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T06:56:40.335627+0000","last_undegraded":"2024-05-27T06:56:40.335627+0000","last_fullsized":"2024-05-27T06:56:40.335627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400511+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T06:56:40.400511+0000","last_peered":"2024-05-27T06:56:40.400511+0000","last_clean":"2024-05-27T06:56:40.400511+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T06:56:40.400511+0000","last_undegraded":"2024-05-27T06:56:40.400511+0000","last_fullsized":"2024-05-27T06:56:40.400511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400638+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T06:56:40.400638+0000","last_peered":"2024-05-27T06:56:40.400638+0000","last_clean":"2024-05-27T06:56:40.400638+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T06:56:40.400638+0000","last_undegraded":"2024-05-27T06:56:40.400638+0000","last_fullsized":"2024-05-27T06:56:40.400638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394546+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T06:56:40.394546+0000","last_peered":"2024-05-27T06:56:40.394546+0000","last_clean":"2024-05-27T06:56:40.394546+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T06:56:40.394546+0000","last_undegraded":"2024-05-27T06:56:40.394546+0000","last_fullsized":"2024-05-27T06:56:40.394546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408290+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T06:56:40.408290+0000","last_peered":"2024-05-27T06:56:40.408290+0000","last_clean":"2024-05-27T06:56:40.408290+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T06:56:40.408290+0000","last_undegraded":"2024-05-27T06:56:40.408290+0000","last_fullsized":"2024-05-27T06:56:40.408290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401656+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T06:56:40.401656+0000","last_peered":"2024-05-27T06:56:40.401656+0000","last_clean":"2024-05-27T06:56:40.401656+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T06:56:40.401656+0000","last_undegraded":"2024-05-27T06:56:40.401656+0000","last_fullsized":"2024-05-27T06:56:40.401656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.350094+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T06:56:40.350094+0000","last_peered":"2024-05-27T06:56:40.350094+0000","last_clean":"2024-05-27T06:56:40.350094+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T06:56:40.350094+0000","last_undegraded":"2024-05-27T06:56:40.350094+0000","last_fullsized":"2024-05-27T06:56:40.350094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.410046+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T06:56:40.410046+0000","last_peered":"2024-05-27T06:56:40.410046+0000","last_clean":"2024-05-27T06:56:40.410046+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T06:56:40.410046+0000","last_undegraded":"2024-05-27T06:56:40.410046+0000","last_fullsized":"2024-05-27T06:56:40.410046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":96,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:19.562961+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T06:57:19.562961+0000","last_peered":"2024-05-27T06:57:19.562961+0000","last_clean":"2024-05-27T06:57:19.562961+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T06:57:19.562961+0000","last_undegraded":"2024-05-27T06:57:19.562961+0000","last_fullsized":"2024-05-27T06:57:19.562961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404777+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T06:56:40.404777+0000","last_peered":"2024-05-27T06:56:40.404777+0000","last_clean":"2024-05-27T06:56:40.404777+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T06:56:40.404777+0000","last_undegraded":"2024-05-27T06:56:40.404777+0000","last_fullsized":"2024-05-27T06:56:40.404777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.407895+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T06:56:40.407895+0000","last_peered":"2024-05-27T06:56:40.407895+0000","last_clean":"2024-05-27T06:56:40.407895+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T06:56:40.407895+0000","last_undegraded":"2024-05-27T06:56:40.407895+0000","last_fullsized":"2024-05-27T06:56:40.407895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.411651+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T06:56:40.411651+0000","last_peered":"2024-05-27T06:56:40.411651+0000","last_clean":"2024-05-27T06:56:40.411651+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T06:56:40.411651+0000","last_undegraded":"2024-05-27T06:56:40.411651+0000","last_fullsized":"2024-05-27T06:56:40.411651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399697+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T06:56:40.399697+0000","last_peered":"2024-05-27T06:56:40.399697+0000","last_clean":"2024-05-27T06:56:40.399697+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T06:56:40.399697+0000","last_undegraded":"2024-05-27T06:56:40.399697+0000","last_fullsized":"2024-05-27T06:56:40.399697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341613+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T06:56:40.341613+0000","last_peered":"2024-05-27T06:56:40.341613+0000","last_clean":"2024-05-27T06:56:40.341613+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T06:56:40.341613+0000","last_undegraded":"2024-05-27T06:56:40.341613+0000","last_fullsized":"2024-05-27T06:56:40.341613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:43.423722+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T06:56:43.423722+0000","last_peered":"2024-05-27T06:56:43.423722+0000","last_clean":"2024-05-27T06:56:43.423722+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T06:56:43.423722+0000","last_undegraded":"2024-05-27T06:56:43.423722+0000","last_fullsized":"2024-05-27T06:56:43.423722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396404+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T06:56:40.396404+0000","last_peered":"2024-05-27T06:56:40.396404+0000","last_clean":"2024-05-27T06:56:40.396404+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T06:56:40.396404+0000","last_undegraded":"2024-05-27T06:56:40.396404+0000","last_fullsized":"2024-05-27T06:56:40.396404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.407803+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T06:56:40.407803+0000","last_peered":"2024-05-27T06:56:40.407803+0000","last_clean":"2024-05-27T06:56:40.407803+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T06:56:40.407803+0000","last_undegraded":"2024-05-27T06:56:40.407803+0000","last_fullsized":"2024-05-27T06:56:40.407803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.433330+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T06:56:42.433330+0000","last_peered":"2024-05-27T06:56:42.433330+0000","last_clean":"2024-05-27T06:56:42.433330+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T06:56:42.433330+0000","last_undegraded":"2024-05-27T06:56:42.433330+0000","last_fullsized":"2024-05-27T06:56:42.433330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341388+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T06:56:40.341388+0000","last_peered":"2024-05-27T06:56:40.341388+0000","last_clean":"2024-05-27T06:56:40.341388+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T06:56:40.341388+0000","last_undegraded":"2024-05-27T06:56:40.341388+0000","last_fullsized":"2024-05-27T06:56:40.341388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349485+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T06:56:40.349485+0000","last_peered":"2024-05-27T06:56:40.349485+0000","last_clean":"2024-05-27T06:56:40.349485+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T06:56:40.349485+0000","last_undegraded":"2024-05-27T06:56:40.349485+0000","last_fullsized":"2024-05-27T06:56:40.349485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:20.571750+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T06:57:20.571750+0000","last_peered":"2024-05-27T06:57:20.571750+0000","last_clean":"2024-05-27T06:57:20.571750+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T06:57:20.571750+0000","last_undegraded":"2024-05-27T06:57:20.571750+0000","last_fullsized":"2024-05-27T06:57:20.571750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400902+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T06:56:40.400902+0000","last_peered":"2024-05-27T06:56:40.400902+0000","last_clean":"2024-05-27T06:56:40.400902+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T06:56:40.400902+0000","last_undegraded":"2024-05-27T06:56:40.400902+0000","last_fullsized":"2024-05-27T06:56:40.400902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349049+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T06:56:40.349049+0000","last_peered":"2024-05-27T06:56:40.349049+0000","last_clean":"2024-05-27T06:56:40.349049+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T06:56:40.349049+0000","last_undegraded":"2024-05-27T06:56:40.349049+0000","last_fullsized":"2024-05-27T06:56:40.349049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.421995+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T06:56:41.421995+0000","last_peered":"2024-05-27T06:56:41.421995+0000","last_clean":"2024-05-27T06:56:41.421995+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T06:56:41.421995+0000","last_undegraded":"2024-05-27T06:56:41.421995+0000","last_fullsized":"2024-05-27T06:56:41.421995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401736+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T06:56:40.401736+0000","last_peered":"2024-05-27T06:56:40.401736+0000","last_clean":"2024-05-27T06:56:40.401736+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T06:56:40.401736+0000","last_undegraded":"2024-05-27T06:56:40.401736+0000","last_fullsized":"2024-05-27T06:56:40.401736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408090+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T06:56:40.408090+0000","last_peered":"2024-05-27T06:56:40.408090+0000","last_clean":"2024-05-27T06:56:40.408090+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T06:56:40.408090+0000","last_undegraded":"2024-05-27T06:56:40.408090+0000","last_fullsized":"2024-05-27T06:56:40.408090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400695+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T06:56:40.400695+0000","last_peered":"2024-05-27T06:56:40.400695+0000","last_clean":"2024-05-27T06:56:40.400695+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T06:56:40.400695+0000","last_undegraded":"2024-05-27T06:56:40.400695+0000","last_fullsized":"2024-05-27T06:56:40.400695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404087+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T06:56:40.404087+0000","last_peered":"2024-05-27T06:56:40.404087+0000","last_clean":"2024-05-27T06:56:40.404087+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T06:56:40.404087+0000","last_undegraded":"2024-05-27T06:56:40.404087+0000","last_fullsized":"2024-05-27T06:56:40.404087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399464+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T06:56:40.399464+0000","last_peered":"2024-05-27T06:56:40.399464+0000","last_clean":"2024-05-27T06:56:40.399464+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T06:56:40.399464+0000","last_undegraded":"2024-05-27T06:56:40.399464+0000","last_fullsized":"2024-05-27T06:56:40.399464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434099+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T06:56:42.434099+0000","last_peered":"2024-05-27T06:56:42.434099+0000","last_clean":"2024-05-27T06:56:42.434099+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T06:56:42.434099+0000","last_undegraded":"2024-05-27T06:56:42.434099+0000","last_fullsized":"2024-05-27T06:56:42.434099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399235+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T06:56:40.399235+0000","last_peered":"2024-05-27T06:56:40.399235+0000","last_clean":"2024-05-27T06:56:40.399235+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T06:56:40.399235+0000","last_undegraded":"2024-05-27T06:56:40.399235+0000","last_fullsized":"2024-05-27T06:56:40.399235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402717+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T06:56:40.402717+0000","last_peered":"2024-05-27T06:56:40.402717+0000","last_clean":"2024-05-27T06:56:40.402717+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T06:56:40.402717+0000","last_undegraded":"2024-05-27T06:56:40.402717+0000","last_fullsized":"2024-05-27T06:56:40.402717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:17.560340+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T06:57:17.560340+0000","last_peered":"2024-05-27T06:57:17.560340+0000","last_clean":"2024-05-27T06:57:17.560340+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T06:57:17.560340+0000","last_undegraded":"2024-05-27T06:57:17.560340+0000","last_fullsized":"2024-05-27T06:57:17.560340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396731+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T06:56:40.396731+0000","last_peered":"2024-05-27T06:56:40.396731+0000","last_clean":"2024-05-27T06:56:40.396731+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T06:56:40.396731+0000","last_undegraded":"2024-05-27T06:56:40.396731+0000","last_fullsized":"2024-05-27T06:56:40.396731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398553+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T06:56:40.398553+0000","last_peered":"2024-05-27T06:56:40.398553+0000","last_clean":"2024-05-27T06:56:40.398553+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T06:56:40.398553+0000","last_undegraded":"2024-05-27T06:56:40.398553+0000","last_fullsized":"2024-05-27T06:56:40.398553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432920+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T06:56:42.432920+0000","last_peered":"2024-05-27T06:56:42.432920+0000","last_clean":"2024-05-27T06:56:42.432920+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T06:56:42.432920+0000","last_undegraded":"2024-05-27T06:56:42.432920+0000","last_fullsized":"2024-05-27T06:56:42.432920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399521+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T06:56:40.399521+0000","last_peered":"2024-05-27T06:56:40.399521+0000","last_clean":"2024-05-27T06:56:40.399521+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T06:56:40.399521+0000","last_undegraded":"2024-05-27T06:56:40.399521+0000","last_fullsized":"2024-05-27T06:56:40.399521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342345+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T06:56:40.342345+0000","last_peered":"2024-05-27T06:56:40.342345+0000","last_clean":"2024-05-27T06:56:40.342345+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T06:56:40.342345+0000","last_undegraded":"2024-05-27T06:56:40.342345+0000","last_fullsized":"2024-05-27T06:56:40.342345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422575+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T06:56:41.422575+0000","last_peered":"2024-05-27T06:56:41.422575+0000","last_clean":"2024-05-27T06:56:41.422575+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T06:56:41.422575+0000","last_undegraded":"2024-05-27T06:56:41.422575+0000","last_fullsized":"2024-05-27T06:56:41.422575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396556+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T06:56:40.396556+0000","last_peered":"2024-05-27T06:56:40.396556+0000","last_clean":"2024-05-27T06:56:40.396556+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T06:56:40.396556+0000","last_undegraded":"2024-05-27T06:56:40.396556+0000","last_fullsized":"2024-05-27T06:56:40.396556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399765+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T06:56:40.399765+0000","last_peered":"2024-05-27T06:56:40.399765+0000","last_clean":"2024-05-27T06:56:40.399765+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T06:56:40.399765+0000","last_undegraded":"2024-05-27T06:56:40.399765+0000","last_fullsized":"2024-05-27T06:56:40.399765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":173,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:17.580264+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T06:57:17.580264+0000","last_peered":"2024-05-27T06:57:17.580264+0000","last_clean":"2024-05-27T06:57:17.580264+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T06:57:17.580264+0000","last_undegraded":"2024-05-27T06:57:17.580264+0000","last_fullsized":"2024-05-27T06:57:17.580264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":78,"num_read_kb":99,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404127+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T06:56:40.404127+0000","last_peered":"2024-05-27T06:56:40.404127+0000","last_clean":"2024-05-27T06:56:40.404127+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T06:56:40.404127+0000","last_undegraded":"2024-05-27T06:56:40.404127+0000","last_fullsized":"2024-05-27T06:56:40.404127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342124+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T06:56:40.342124+0000","last_peered":"2024-05-27T06:56:40.342124+0000","last_clean":"2024-05-27T06:56:40.342124+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T06:56:40.342124+0000","last_undegraded":"2024-05-27T06:56:40.342124+0000","last_fullsized":"2024-05-27T06:56:40.342124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449381+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T06:56:44.449381+0000","last_peered":"2024-05-27T06:56:44.449381+0000","last_clean":"2024-05-27T06:56:44.449381+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T06:56:44.449381+0000","last_undegraded":"2024-05-27T06:56:44.449381+0000","last_fullsized":"2024-05-27T06:56:44.449381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335193+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T06:56:40.335193+0000","last_peered":"2024-05-27T06:56:40.335193+0000","last_clean":"2024-05-27T06:56:40.335193+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T06:56:40.335193+0000","last_undegraded":"2024-05-27T06:56:40.335193+0000","last_fullsized":"2024-05-27T06:56:40.335193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401346+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T06:56:40.401346+0000","last_peered":"2024-05-27T06:56:40.401346+0000","last_clean":"2024-05-27T06:56:40.401346+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T06:56:40.401346+0000","last_undegraded":"2024-05-27T06:56:40.401346+0000","last_fullsized":"2024-05-27T06:56:40.401346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434087+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T06:56:42.434087+0000","last_peered":"2024-05-27T06:56:42.434087+0000","last_clean":"2024-05-27T06:56:42.434087+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T06:56:42.434087+0000","last_undegraded":"2024-05-27T06:56:42.434087+0000","last_fullsized":"2024-05-27T06:56:42.434087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338964+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T06:56:40.338964+0000","last_peered":"2024-05-27T06:56:40.338964+0000","last_clean":"2024-05-27T06:56:40.338964+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T06:56:40.338964+0000","last_undegraded":"2024-05-27T06:56:40.338964+0000","last_fullsized":"2024-05-27T06:56:40.338964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402073+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T06:56:40.402073+0000","last_peered":"2024-05-27T06:56:40.402073+0000","last_clean":"2024-05-27T06:56:40.402073+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T06:56:40.402073+0000","last_undegraded":"2024-05-27T06:56:40.402073+0000","last_fullsized":"2024-05-27T06:56:40.402073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:20.577249+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T06:57:20.577249+0000","last_peered":"2024-05-27T06:57:20.577249+0000","last_clean":"2024-05-27T06:57:20.577249+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T06:57:20.577249+0000","last_undegraded":"2024-05-27T06:57:20.577249+0000","last_fullsized":"2024-05-27T06:57:20.577249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338007+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T06:56:40.338007+0000","last_peered":"2024-05-27T06:56:40.338007+0000","last_clean":"2024-05-27T06:56:40.338007+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T06:56:40.338007+0000","last_undegraded":"2024-05-27T06:56:40.338007+0000","last_fullsized":"2024-05-27T06:56:40.338007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335370+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T06:56:40.335370+0000","last_peered":"2024-05-27T06:56:40.335370+0000","last_clean":"2024-05-27T06:56:40.335370+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T06:56:40.335370+0000","last_undegraded":"2024-05-27T06:56:40.335370+0000","last_fullsized":"2024-05-27T06:56:40.335370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432272+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T06:56:42.432272+0000","last_peered":"2024-05-27T06:56:42.432272+0000","last_clean":"2024-05-27T06:56:42.432272+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T06:56:42.432272+0000","last_undegraded":"2024-05-27T06:56:42.432272+0000","last_fullsized":"2024-05-27T06:56:42.432272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340408+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T06:56:40.340408+0000","last_peered":"2024-05-27T06:56:40.340408+0000","last_clean":"2024-05-27T06:56:40.340408+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T06:56:40.340408+0000","last_undegraded":"2024-05-27T06:56:40.340408+0000","last_fullsized":"2024-05-27T06:56:40.340408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340047+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T06:56:40.340047+0000","last_peered":"2024-05-27T06:56:40.340047+0000","last_clean":"2024-05-27T06:56:40.340047+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T06:56:40.340047+0000","last_undegraded":"2024-05-27T06:56:40.340047+0000","last_fullsized":"2024-05-27T06:56:40.340047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451292+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T06:56:44.451292+0000","last_peered":"2024-05-27T06:56:44.451292+0000","last_clean":"2024-05-27T06:56:44.451292+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T06:56:44.451292+0000","last_undegraded":"2024-05-27T06:56:44.451292+0000","last_fullsized":"2024-05-27T06:56:44.451292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.407813+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T06:56:40.407813+0000","last_peered":"2024-05-27T06:56:40.407813+0000","last_clean":"2024-05-27T06:56:40.407813+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T06:56:40.407813+0000","last_undegraded":"2024-05-27T06:56:40.407813+0000","last_fullsized":"2024-05-27T06:56:40.407813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401367+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T06:56:40.401367+0000","last_peered":"2024-05-27T06:56:40.401367+0000","last_clean":"2024-05-27T06:56:40.401367+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T06:56:40.401367+0000","last_undegraded":"2024-05-27T06:56:40.401367+0000","last_fullsized":"2024-05-27T06:56:40.401367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422923+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T06:56:41.422923+0000","last_peered":"2024-05-27T06:56:41.422923+0000","last_clean":"2024-05-27T06:56:41.422923+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T06:56:41.422923+0000","last_undegraded":"2024-05-27T06:56:41.422923+0000","last_fullsized":"2024-05-27T06:56:41.422923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394931+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T06:56:40.394931+0000","last_peered":"2024-05-27T06:56:40.394931+0000","last_clean":"2024-05-27T06:56:40.394931+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T06:56:40.394931+0000","last_undegraded":"2024-05-27T06:56:40.394931+0000","last_fullsized":"2024-05-27T06:56:40.394931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335765+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T06:56:40.335765+0000","last_peered":"2024-05-27T06:56:40.335765+0000","last_clean":"2024-05-27T06:56:40.335765+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T06:56:40.335765+0000","last_undegraded":"2024-05-27T06:56:40.335765+0000","last_fullsized":"2024-05-27T06:56:40.335765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:18.561469+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T06:57:18.561469+0000","last_peered":"2024-05-27T06:57:18.561469+0000","last_clean":"2024-05-27T06:57:18.561469+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T06:57:18.561469+0000","last_undegraded":"2024-05-27T06:57:18.561469+0000","last_fullsized":"2024-05-27T06:57:18.561469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.393918+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T06:56:40.393918+0000","last_peered":"2024-05-27T06:56:40.393918+0000","last_clean":"2024-05-27T06:56:40.393918+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T06:56:40.393918+0000","last_undegraded":"2024-05-27T06:56:40.393918+0000","last_fullsized":"2024-05-27T06:56:40.393918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335014+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T06:56:40.335014+0000","last_peered":"2024-05-27T06:56:40.335014+0000","last_clean":"2024-05-27T06:56:40.335014+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T06:56:40.335014+0000","last_undegraded":"2024-05-27T06:56:40.335014+0000","last_fullsized":"2024-05-27T06:56:40.335014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340613+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T06:56:40.340613+0000","last_peered":"2024-05-27T06:56:40.340613+0000","last_clean":"2024-05-27T06:56:40.340613+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T06:56:40.340613+0000","last_undegraded":"2024-05-27T06:56:40.340613+0000","last_fullsized":"2024-05-27T06:56:40.340613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:15.560294+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T06:57:15.560294+0000","last_peered":"2024-05-27T06:57:15.560294+0000","last_clean":"2024-05-27T06:57:15.560294+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T06:57:15.560294+0000","last_undegraded":"2024-05-27T06:57:15.560294+0000","last_fullsized":"2024-05-27T06:57:15.560294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339695+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T06:56:40.339695+0000","last_peered":"2024-05-27T06:56:40.339695+0000","last_clean":"2024-05-27T06:56:40.339695+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T06:56:40.339695+0000","last_undegraded":"2024-05-27T06:56:40.339695+0000","last_fullsized":"2024-05-27T06:56:40.339695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.405949+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T06:56:40.405949+0000","last_peered":"2024-05-27T06:56:40.405949+0000","last_clean":"2024-05-27T06:56:40.405949+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T06:56:40.405949+0000","last_undegraded":"2024-05-27T06:56:40.405949+0000","last_fullsized":"2024-05-27T06:56:40.405949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450805+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T06:56:44.450805+0000","last_peered":"2024-05-27T06:56:44.450805+0000","last_clean":"2024-05-27T06:56:44.450805+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T06:56:44.450805+0000","last_undegraded":"2024-05-27T06:56:44.450805+0000","last_fullsized":"2024-05-27T06:56:44.450805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.337815+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T06:56:40.337815+0000","last_peered":"2024-05-27T06:56:40.337815+0000","last_clean":"2024-05-27T06:56:40.337815+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T06:56:40.337815+0000","last_undegraded":"2024-05-27T06:56:40.337815+0000","last_fullsized":"2024-05-27T06:56:40.337815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342055+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T06:56:40.342055+0000","last_peered":"2024-05-27T06:56:40.342055+0000","last_clean":"2024-05-27T06:56:40.342055+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T06:56:40.342055+0000","last_undegraded":"2024-05-27T06:56:40.342055+0000","last_fullsized":"2024-05-27T06:56:40.342055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.334716+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T06:56:40.334716+0000","last_peered":"2024-05-27T06:56:40.334716+0000","last_clean":"2024-05-27T06:56:40.334716+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T06:56:40.334716+0000","last_undegraded":"2024-05-27T06:56:40.334716+0000","last_fullsized":"2024-05-27T06:56:40.334716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432537+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T06:56:42.432537+0000","last_peered":"2024-05-27T06:56:42.432537+0000","last_clean":"2024-05-27T06:56:42.432537+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T06:56:42.432537+0000","last_undegraded":"2024-05-27T06:56:42.432537+0000","last_fullsized":"2024-05-27T06:56:42.432537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402420+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T06:56:40.402420+0000","last_peered":"2024-05-27T06:56:40.402420+0000","last_clean":"2024-05-27T06:56:40.402420+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T06:56:40.402420+0000","last_undegraded":"2024-05-27T06:56:40.402420+0000","last_fullsized":"2024-05-27T06:56:40.402420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450385+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T06:56:44.450385+0000","last_peered":"2024-05-27T06:56:44.450385+0000","last_clean":"2024-05-27T06:56:44.450385+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T06:56:44.450385+0000","last_undegraded":"2024-05-27T06:56:44.450385+0000","last_fullsized":"2024-05-27T06:56:44.450385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335904+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T06:56:40.335904+0000","last_peered":"2024-05-27T06:56:40.335904+0000","last_clean":"2024-05-27T06:56:40.335904+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T06:56:40.335904+0000","last_undegraded":"2024-05-27T06:56:40.335904+0000","last_fullsized":"2024-05-27T06:56:40.335904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.403943+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T06:56:40.403943+0000","last_peered":"2024-05-27T06:56:40.403943+0000","last_clean":"2024-05-27T06:56:40.403943+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T06:56:40.403943+0000","last_undegraded":"2024-05-27T06:56:40.403943+0000","last_fullsized":"2024-05-27T06:56:40.403943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:16.307934+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T06:57:16.307934+0000","last_peered":"2024-05-27T06:57:16.307934+0000","last_clean":"2024-05-27T06:57:16.307934+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T06:57:16.307934+0000","last_undegraded":"2024-05-27T06:57:16.307934+0000","last_fullsized":"2024-05-27T06:57:16.307934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402135+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T06:56:40.402135+0000","last_peered":"2024-05-27T06:56:40.402135+0000","last_clean":"2024-05-27T06:56:40.402135+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T06:56:40.402135+0000","last_undegraded":"2024-05-27T06:56:40.402135+0000","last_fullsized":"2024-05-27T06:56:40.402135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348176+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T06:56:40.348176+0000","last_peered":"2024-05-27T06:56:40.348176+0000","last_clean":"2024-05-27T06:56:40.348176+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T06:56:40.348176+0000","last_undegraded":"2024-05-27T06:56:40.348176+0000","last_fullsized":"2024-05-27T06:56:40.348176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449227+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T06:56:44.449227+0000","last_peered":"2024-05-27T06:56:44.449227+0000","last_clean":"2024-05-27T06:56:44.449227+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T06:56:44.449227+0000","last_undegraded":"2024-05-27T06:56:44.449227+0000","last_fullsized":"2024-05-27T06:56:44.449227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399656+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T06:56:40.399656+0000","last_peered":"2024-05-27T06:56:40.399656+0000","last_clean":"2024-05-27T06:56:40.399656+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T06:56:40.399656+0000","last_undegraded":"2024-05-27T06:56:40.399656+0000","last_fullsized":"2024-05-27T06:56:40.399656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344023+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T06:56:40.344023+0000","last_peered":"2024-05-27T06:56:40.344023+0000","last_clean":"2024-05-27T06:56:40.344023+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T06:56:40.344023+0000","last_undegraded":"2024-05-27T06:56:40.344023+0000","last_fullsized":"2024-05-27T06:56:40.344023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394827+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T06:56:40.394827+0000","last_peered":"2024-05-27T06:56:40.394827+0000","last_clean":"2024-05-27T06:56:40.394827+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T06:56:40.394827+0000","last_undegraded":"2024-05-27T06:56:40.394827+0000","last_fullsized":"2024-05-27T06:56:40.394827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400252+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T06:56:40.400252+0000","last_peered":"2024-05-27T06:56:40.400252+0000","last_clean":"2024-05-27T06:56:40.400252+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T06:56:40.400252+0000","last_undegraded":"2024-05-27T06:56:40.400252+0000","last_fullsized":"2024-05-27T06:56:40.400252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402480+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T06:56:40.402480+0000","last_peered":"2024-05-27T06:56:40.402480+0000","last_clean":"2024-05-27T06:56:40.402480+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T06:56:40.402480+0000","last_undegraded":"2024-05-27T06:56:40.402480+0000","last_fullsized":"2024-05-27T06:56:40.402480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434757+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T06:56:42.434757+0000","last_peered":"2024-05-27T06:56:42.434757+0000","last_clean":"2024-05-27T06:56:42.434757+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T06:56:42.434757+0000","last_undegraded":"2024-05-27T06:56:42.434757+0000","last_fullsized":"2024-05-27T06:56:42.434757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399660+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T06:56:40.399660+0000","last_peered":"2024-05-27T06:56:40.399660+0000","last_clean":"2024-05-27T06:56:40.399660+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T06:56:40.399660+0000","last_undegraded":"2024-05-27T06:56:40.399660+0000","last_fullsized":"2024-05-27T06:56:40.399660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.343231+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T06:56:40.343231+0000","last_peered":"2024-05-27T06:56:40.343231+0000","last_clean":"2024-05-27T06:56:40.343231+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T06:56:40.343231+0000","last_undegraded":"2024-05-27T06:56:40.343231+0000","last_fullsized":"2024-05-27T06:56:40.343231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.407953+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T06:56:42.407953+0000","last_peered":"2024-05-27T06:56:42.407953+0000","last_clean":"2024-05-27T06:56:42.407953+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T06:56:42.407953+0000","last_undegraded":"2024-05-27T06:56:42.407953+0000","last_fullsized":"2024-05-27T06:56:42.407953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.403915+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T06:56:40.403915+0000","last_peered":"2024-05-27T06:56:40.403915+0000","last_clean":"2024-05-27T06:56:40.403915+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T06:56:40.403915+0000","last_undegraded":"2024-05-27T06:56:40.403915+0000","last_fullsized":"2024-05-27T06:56:40.403915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.343929+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T06:56:40.343929+0000","last_peered":"2024-05-27T06:56:40.343929+0000","last_clean":"2024-05-27T06:56:40.343929+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T06:56:40.343929+0000","last_undegraded":"2024-05-27T06:56:40.343929+0000","last_fullsized":"2024-05-27T06:56:40.343929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451041+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T06:56:44.451041+0000","last_peered":"2024-05-27T06:56:44.451041+0000","last_clean":"2024-05-27T06:56:44.451041+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T06:56:44.451041+0000","last_undegraded":"2024-05-27T06:56:44.451041+0000","last_fullsized":"2024-05-27T06:56:44.451041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338725+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T06:56:40.338725+0000","last_peered":"2024-05-27T06:56:40.338725+0000","last_clean":"2024-05-27T06:56:40.338725+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T06:56:40.338725+0000","last_undegraded":"2024-05-27T06:56:40.338725+0000","last_fullsized":"2024-05-27T06:56:40.338725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396142+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T06:56:40.396142+0000","last_peered":"2024-05-27T06:56:40.396142+0000","last_clean":"2024-05-27T06:56:40.396142+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T06:56:40.396142+0000","last_undegraded":"2024-05-27T06:56:40.396142+0000","last_fullsized":"2024-05-27T06:56:40.396142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:17.562096+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T06:57:17.562096+0000","last_peered":"2024-05-27T06:57:17.562096+0000","last_clean":"2024-05-27T06:57:17.562096+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T06:57:17.562096+0000","last_undegraded":"2024-05-27T06:57:17.562096+0000","last_fullsized":"2024-05-27T06:57:17.562096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340444+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T06:56:40.340444+0000","last_peered":"2024-05-27T06:56:40.340444+0000","last_clean":"2024-05-27T06:56:40.340444+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T06:56:40.340444+0000","last_undegraded":"2024-05-27T06:56:40.340444+0000","last_fullsized":"2024-05-27T06:56:40.340444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398710+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T06:56:40.398710+0000","last_peered":"2024-05-27T06:56:40.398710+0000","last_clean":"2024-05-27T06:56:40.398710+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T06:56:40.398710+0000","last_undegraded":"2024-05-27T06:56:40.398710+0000","last_fullsized":"2024-05-27T06:56:40.398710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451295+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T06:56:44.451295+0000","last_peered":"2024-05-27T06:56:44.451295+0000","last_clean":"2024-05-27T06:56:44.451295+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T06:56:44.451295+0000","last_undegraded":"2024-05-27T06:56:44.451295+0000","last_fullsized":"2024-05-27T06:56:44.451295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339288+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T06:56:40.339288+0000","last_peered":"2024-05-27T06:56:40.339288+0000","last_clean":"2024-05-27T06:56:40.339288+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T06:56:40.339288+0000","last_undegraded":"2024-05-27T06:56:40.339288+0000","last_fullsized":"2024-05-27T06:56:40.339288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339644+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T06:56:40.339644+0000","last_peered":"2024-05-27T06:56:40.339644+0000","last_clean":"2024-05-27T06:56:40.339644+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T06:56:40.339644+0000","last_undegraded":"2024-05-27T06:56:40.339644+0000","last_fullsized":"2024-05-27T06:56:40.339644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.430901+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T06:56:42.430901+0000","last_peered":"2024-05-27T06:56:42.430901+0000","last_clean":"2024-05-27T06:56:42.430901+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T06:56:42.430901+0000","last_undegraded":"2024-05-27T06:56:42.430901+0000","last_fullsized":"2024-05-27T06:56:42.430901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340380+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T06:56:40.340380+0000","last_peered":"2024-05-27T06:56:40.340380+0000","last_clean":"2024-05-27T06:56:40.340380+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T06:56:40.340380+0000","last_undegraded":"2024-05-27T06:56:40.340380+0000","last_fullsized":"2024-05-27T06:56:40.340380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400049+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T06:56:40.400049+0000","last_peered":"2024-05-27T06:56:40.400049+0000","last_clean":"2024-05-27T06:56:40.400049+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T06:56:40.400049+0000","last_undegraded":"2024-05-27T06:56:40.400049+0000","last_fullsized":"2024-05-27T06:56:40.400049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348904+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T06:56:40.348904+0000","last_peered":"2024-05-27T06:56:40.348904+0000","last_clean":"2024-05-27T06:56:40.348904+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T06:56:40.348904+0000","last_undegraded":"2024-05-27T06:56:40.348904+0000","last_fullsized":"2024-05-27T06:56:40.348904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349077+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T06:56:40.349077+0000","last_peered":"2024-05-27T06:56:40.349077+0000","last_clean":"2024-05-27T06:56:40.349077+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T06:56:40.349077+0000","last_undegraded":"2024-05-27T06:56:40.349077+0000","last_fullsized":"2024-05-27T06:56:40.349077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399697+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T06:56:40.399697+0000","last_peered":"2024-05-27T06:56:40.399697+0000","last_clean":"2024-05-27T06:56:40.399697+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T06:56:40.399697+0000","last_undegraded":"2024-05-27T06:56:40.399697+0000","last_fullsized":"2024-05-27T06:56:40.399697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335968+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T06:56:40.335968+0000","last_peered":"2024-05-27T06:56:40.335968+0000","last_clean":"2024-05-27T06:56:40.335968+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T06:56:40.335968+0000","last_undegraded":"2024-05-27T06:56:40.335968+0000","last_fullsized":"2024-05-27T06:56:40.335968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.405964+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T06:56:40.405964+0000","last_peered":"2024-05-27T06:56:40.405964+0000","last_clean":"2024-05-27T06:56:40.405964+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T06:56:40.405964+0000","last_undegraded":"2024-05-27T06:56:40.405964+0000","last_fullsized":"2024-05-27T06:56:40.405964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349998+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T06:56:40.349998+0000","last_peered":"2024-05-27T06:56:40.349998+0000","last_clean":"2024-05-27T06:56:40.349998+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T06:56:40.349998+0000","last_undegraded":"2024-05-27T06:56:40.349998+0000","last_fullsized":"2024-05-27T06:56:40.349998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402488+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T06:56:40.402488+0000","last_peered":"2024-05-27T06:56:40.402488+0000","last_clean":"2024-05-27T06:56:40.402488+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T06:56:40.402488+0000","last_undegraded":"2024-05-27T06:56:40.402488+0000","last_fullsized":"2024-05-27T06:56:40.402488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.334548+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T06:56:40.334548+0000","last_peered":"2024-05-27T06:56:40.334548+0000","last_clean":"2024-05-27T06:56:40.334548+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T06:56:40.334548+0000","last_undegraded":"2024-05-27T06:56:40.334548+0000","last_fullsized":"2024-05-27T06:56:40.334548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402965+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T06:56:40.402965+0000","last_peered":"2024-05-27T06:56:40.402965+0000","last_clean":"2024-05-27T06:56:40.402965+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T06:56:40.402965+0000","last_undegraded":"2024-05-27T06:56:40.402965+0000","last_fullsized":"2024-05-27T06:56:40.402965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396912+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T06:56:40.396912+0000","last_peered":"2024-05-27T06:56:40.396912+0000","last_clean":"2024-05-27T06:56:40.396912+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T06:56:40.396912+0000","last_undegraded":"2024-05-27T06:56:40.396912+0000","last_fullsized":"2024-05-27T06:56:40.396912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335453+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T06:56:40.335453+0000","last_peered":"2024-05-27T06:56:40.335453+0000","last_clean":"2024-05-27T06:56:40.335453+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T06:56:40.335453+0000","last_undegraded":"2024-05-27T06:56:40.335453+0000","last_fullsized":"2024-05-27T06:56:40.335453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401971+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T06:56:40.401971+0000","last_peered":"2024-05-27T06:56:40.401971+0000","last_clean":"2024-05-27T06:56:40.401971+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T06:56:40.401971+0000","last_undegraded":"2024-05-27T06:56:40.401971+0000","last_fullsized":"2024-05-27T06:56:40.401971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69811,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":396,"num_read_kb":270,"num_write":251,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":217963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678063,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1170492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234052,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":521377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430285,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1662976,"data_stored":1118265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626515,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":516894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":584405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":518350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182505,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986275,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":587449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822748,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T06:57:24.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T06:57:24.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:24 smithi016 ceph-mon[31415]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:24.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:24 smithi164 ceph-mon[35327]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:24.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:24 smithi069 ceph-mon[35276]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:25.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:25 smithi016 ceph-mon[31415]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T06:57:25.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:25 smithi164 ceph-mon[35327]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T06:57:25.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:25 smithi069 ceph-mon[35276]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T06:57:27.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:27 smithi016 ceph-mon[31415]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 -- 172.21.15.16:0/2057338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 msgr2=0x7efe5410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 --2- 172.21.15.16:0/2057338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe5410ad90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efe40009990 tx=0x7efe40009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 -- 172.21.15.16:0/2057338557 shutdown_connections 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 --2- 172.21.15.16:0/2057338557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe54117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 --2- 172.21.15.16:0/2057338557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 0x7efe54110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 --2- 172.21.15.16:0/2057338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe5410ad90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 -- 172.21.15.16:0/2057338557 >> 172.21.15.16:0/2057338557 conn(0x7efe54077c70 msgr2=0x7efe540780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 -- 172.21.15.16:0/2057338557 shutdown_connections 2024-05-27T06:57:27.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.393+0000 7efe5ac00700 1 -- 172.21.15.16:0/2057338557 wait complete. 2024-05-27T06:57:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.394+0000 7efe5ac00700 1 Processor -- start 2024-05-27T06:57:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.394+0000 7efe5ac00700 1 -- start start 2024-05-27T06:57:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe541a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 0x7efe541a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe5411aa60 con 0x7efe5410a980 2024-05-27T06:57:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe5411a8e0 con 0x7efe5410b2d0 2024-05-27T06:57:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe5ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe5411abe0 con 0x7efe54110960 2024-05-27T06:57:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe53e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe53e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55270/0 (socket says 172.21.15.16:55270) 2024-05-27T06:57:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe53e00700 1 -- 172.21.15.16:0/3586719700 learned_addr learned my addr 172.21.15.16:0/3586719700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.395+0000 7efe53400700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe541a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe58c00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 0x7efe541a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 -- 172.21.15.16:0/3586719700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 msgr2=0x7efe541a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 0x7efe541a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 -- 172.21.15.16:0/3586719700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 msgr2=0x7efe541a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe541a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 -- 172.21.15.16:0/3586719700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a63e0 con 0x7efe5410a980 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53400700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe541a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe53e00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b500 tx=0x7efe4400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.396+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44015020 con 0x7efe5410a980 2024-05-27T06:57:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.397+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe44011cc0 con 0x7efe5410a980 2024-05-27T06:57:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.397+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44010c80 con 0x7efe5410a980 2024-05-27T06:57:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.397+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a66d0 con 0x7efe5410a980 2024-05-27T06:57:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.397+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541adff0 con 0x7efe5410a980 2024-05-27T06:57:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.398+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5410c070 con 0x7efe5410a980 2024-05-27T06:57:27.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.401+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efe44018020 con 0x7efe5410a980 2024-05-27T06:57:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.402+0000 7efe50c00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 0x7efe380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.403+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efe440e2440 con 0x7efe5410a980 2024-05-27T06:57:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.403+0000 7efe53400700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 0x7efe380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.404+0000 7efe53400700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 0x7efe380b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b5c0 tx=0x7efe400057e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.406+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe4401f020 con 0x7efe5410a980 2024-05-27T06:57:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:27 smithi164 ceph-mon[35327]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T06:57:27.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:27 smithi069 ceph-mon[35276]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T06:57:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.641+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe541a2870 con 0x7efe380b2910 2024-05-27T06:57:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.652+0000 7efe50c00700 1 -- 172.21.15.16:0/3586719700 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340442 (secure 0 0 0) 0x7efe541a2870 con 0x7efe380b2910 2024-05-27T06:57:27.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.659+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 msgr2=0x7efe380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.659+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 0x7efe380b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b5c0 tx=0x7efe400057e0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.659+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 msgr2=0x7efe541a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.659+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b500 tx=0x7efe4400b8c0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 shutdown_connections 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe380b2910 0x7efe380b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe54110960 0x7efe541a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5410a980 0x7efe541a1510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 --2- 172.21.15.16:0/3586719700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe5410b2d0 0x7efe541a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 >> 172.21.15.16:0/3586719700 conn(0x7efe54077c70 msgr2=0x7efe5407b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 shutdown_connections 2024-05-27T06:57:27.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:27.660+0000 7efe5ac00700 1 -- 172.21.15.16:0/3586719700 wait complete. 2024-05-27T06:57:27.654 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T06:57:28.532 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-05-27T06:57:26.161842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664678,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":513,"num_read_kb":364,"num_write":377,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380192,"kb_used_data":14396,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561632,"statfs":{"total":1151940427776,"available":1151551111168,"internally_reserved":0,"allocated":14741504,"data_stored":8215472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":60,"num_write":8,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408775+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T06:56:40.408775+0000","last_peered":"2024-05-27T06:56:40.408775+0000","last_clean":"2024-05-27T06:56:40.408775+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T06:56:40.408775+0000","last_undegraded":"2024-05-27T06:56:40.408775+0000","last_fullsized":"2024-05-27T06:56:40.408775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399796+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T06:56:40.399796+0000","last_peered":"2024-05-27T06:56:40.399796+0000","last_clean":"2024-05-27T06:56:40.399796+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T06:56:40.399796+0000","last_undegraded":"2024-05-27T06:56:40.399796+0000","last_fullsized":"2024-05-27T06:56:40.399796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399147+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T06:56:40.399147+0000","last_peered":"2024-05-27T06:56:40.399147+0000","last_clean":"2024-05-27T06:56:40.399147+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T06:56:40.399147+0000","last_undegraded":"2024-05-27T06:56:40.399147+0000","last_fullsized":"2024-05-27T06:56:40.399147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341588+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T06:56:40.341588+0000","last_peered":"2024-05-27T06:56:40.341588+0000","last_clean":"2024-05-27T06:56:40.341588+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T06:56:40.341588+0000","last_undegraded":"2024-05-27T06:56:40.341588+0000","last_fullsized":"2024-05-27T06:56:40.341588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401634+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T06:56:40.401634+0000","last_peered":"2024-05-27T06:56:40.401634+0000","last_clean":"2024-05-27T06:56:40.401634+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T06:56:40.401634+0000","last_undegraded":"2024-05-27T06:56:40.401634+0000","last_fullsized":"2024-05-27T06:56:40.401634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.392651+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T06:56:40.392651+0000","last_peered":"2024-05-27T06:56:40.392651+0000","last_clean":"2024-05-27T06:56:40.392651+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T06:56:40.392651+0000","last_undegraded":"2024-05-27T06:56:40.392651+0000","last_fullsized":"2024-05-27T06:56:40.392651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338271+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T06:56:40.338271+0000","last_peered":"2024-05-27T06:56:40.338271+0000","last_clean":"2024-05-27T06:56:40.338271+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T06:56:40.338271+0000","last_undegraded":"2024-05-27T06:56:40.338271+0000","last_fullsized":"2024-05-27T06:56:40.338271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404208+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T06:56:40.404208+0000","last_peered":"2024-05-27T06:56:40.404208+0000","last_clean":"2024-05-27T06:56:40.404208+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T06:56:40.404208+0000","last_undegraded":"2024-05-27T06:56:40.404208+0000","last_fullsized":"2024-05-27T06:56:40.404208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396944+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T06:56:40.396944+0000","last_peered":"2024-05-27T06:56:40.396944+0000","last_clean":"2024-05-27T06:56:40.396944+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T06:56:40.396944+0000","last_undegraded":"2024-05-27T06:56:40.396944+0000","last_fullsized":"2024-05-27T06:56:40.396944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394285+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T06:56:40.394285+0000","last_peered":"2024-05-27T06:56:40.394285+0000","last_clean":"2024-05-27T06:56:40.394285+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T06:56:40.394285+0000","last_undegraded":"2024-05-27T06:56:40.394285+0000","last_fullsized":"2024-05-27T06:56:40.394285+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339992+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T06:56:40.339992+0000","last_peered":"2024-05-27T06:56:40.339992+0000","last_clean":"2024-05-27T06:56:40.339992+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T06:56:40.339992+0000","last_undegraded":"2024-05-27T06:56:40.339992+0000","last_fullsized":"2024-05-27T06:56:40.339992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341754+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T06:56:40.341754+0000","last_peered":"2024-05-27T06:56:40.341754+0000","last_clean":"2024-05-27T06:56:40.341754+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T06:56:40.341754+0000","last_undegraded":"2024-05-27T06:56:40.341754+0000","last_fullsized":"2024-05-27T06:56:40.341754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335627+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T06:56:40.335627+0000","last_peered":"2024-05-27T06:56:40.335627+0000","last_clean":"2024-05-27T06:56:40.335627+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T06:56:40.335627+0000","last_undegraded":"2024-05-27T06:56:40.335627+0000","last_fullsized":"2024-05-27T06:56:40.335627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400511+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T06:56:40.400511+0000","last_peered":"2024-05-27T06:56:40.400511+0000","last_clean":"2024-05-27T06:56:40.400511+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T06:56:40.400511+0000","last_undegraded":"2024-05-27T06:56:40.400511+0000","last_fullsized":"2024-05-27T06:56:40.400511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400638+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T06:56:40.400638+0000","last_peered":"2024-05-27T06:56:40.400638+0000","last_clean":"2024-05-27T06:56:40.400638+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T06:56:40.400638+0000","last_undegraded":"2024-05-27T06:56:40.400638+0000","last_fullsized":"2024-05-27T06:56:40.400638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394546+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T06:56:40.394546+0000","last_peered":"2024-05-27T06:56:40.394546+0000","last_clean":"2024-05-27T06:56:40.394546+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T06:56:40.394546+0000","last_undegraded":"2024-05-27T06:56:40.394546+0000","last_fullsized":"2024-05-27T06:56:40.394546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408290+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T06:56:40.408290+0000","last_peered":"2024-05-27T06:56:40.408290+0000","last_clean":"2024-05-27T06:56:40.408290+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T06:56:40.408290+0000","last_undegraded":"2024-05-27T06:56:40.408290+0000","last_fullsized":"2024-05-27T06:56:40.408290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401656+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T06:56:40.401656+0000","last_peered":"2024-05-27T06:56:40.401656+0000","last_clean":"2024-05-27T06:56:40.401656+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T06:56:40.401656+0000","last_undegraded":"2024-05-27T06:56:40.401656+0000","last_fullsized":"2024-05-27T06:56:40.401656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.350094+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T06:56:40.350094+0000","last_peered":"2024-05-27T06:56:40.350094+0000","last_clean":"2024-05-27T06:56:40.350094+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T06:56:40.350094+0000","last_undegraded":"2024-05-27T06:56:40.350094+0000","last_fullsized":"2024-05-27T06:56:40.350094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.410046+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T06:56:40.410046+0000","last_peered":"2024-05-27T06:56:40.410046+0000","last_clean":"2024-05-27T06:56:40.410046+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T06:56:40.410046+0000","last_undegraded":"2024-05-27T06:56:40.410046+0000","last_fullsized":"2024-05-27T06:56:40.410046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":96,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:19.562961+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T06:57:19.562961+0000","last_peered":"2024-05-27T06:57:19.562961+0000","last_clean":"2024-05-27T06:57:19.562961+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T06:57:19.562961+0000","last_undegraded":"2024-05-27T06:57:19.562961+0000","last_fullsized":"2024-05-27T06:57:19.562961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404777+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T06:56:40.404777+0000","last_peered":"2024-05-27T06:56:40.404777+0000","last_clean":"2024-05-27T06:56:40.404777+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T06:56:40.404777+0000","last_undegraded":"2024-05-27T06:56:40.404777+0000","last_fullsized":"2024-05-27T06:56:40.404777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.407895+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T06:56:40.407895+0000","last_peered":"2024-05-27T06:56:40.407895+0000","last_clean":"2024-05-27T06:56:40.407895+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T06:56:40.407895+0000","last_undegraded":"2024-05-27T06:56:40.407895+0000","last_fullsized":"2024-05-27T06:56:40.407895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.411651+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T06:56:40.411651+0000","last_peered":"2024-05-27T06:56:40.411651+0000","last_clean":"2024-05-27T06:56:40.411651+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T06:56:40.411651+0000","last_undegraded":"2024-05-27T06:56:40.411651+0000","last_fullsized":"2024-05-27T06:56:40.411651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399697+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T06:56:40.399697+0000","last_peered":"2024-05-27T06:56:40.399697+0000","last_clean":"2024-05-27T06:56:40.399697+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T06:56:40.399697+0000","last_undegraded":"2024-05-27T06:56:40.399697+0000","last_fullsized":"2024-05-27T06:56:40.399697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341613+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T06:56:40.341613+0000","last_peered":"2024-05-27T06:56:40.341613+0000","last_clean":"2024-05-27T06:56:40.341613+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T06:56:40.341613+0000","last_undegraded":"2024-05-27T06:56:40.341613+0000","last_fullsized":"2024-05-27T06:56:40.341613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:43.423722+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T06:56:43.423722+0000","last_peered":"2024-05-27T06:56:43.423722+0000","last_clean":"2024-05-27T06:56:43.423722+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T06:56:43.423722+0000","last_undegraded":"2024-05-27T06:56:43.423722+0000","last_fullsized":"2024-05-27T06:56:43.423722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396404+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T06:56:40.396404+0000","last_peered":"2024-05-27T06:56:40.396404+0000","last_clean":"2024-05-27T06:56:40.396404+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T06:56:40.396404+0000","last_undegraded":"2024-05-27T06:56:40.396404+0000","last_fullsized":"2024-05-27T06:56:40.396404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.407803+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T06:56:40.407803+0000","last_peered":"2024-05-27T06:56:40.407803+0000","last_clean":"2024-05-27T06:56:40.407803+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T06:56:40.407803+0000","last_undegraded":"2024-05-27T06:56:40.407803+0000","last_fullsized":"2024-05-27T06:56:40.407803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.433330+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T06:56:42.433330+0000","last_peered":"2024-05-27T06:56:42.433330+0000","last_clean":"2024-05-27T06:56:42.433330+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T06:56:42.433330+0000","last_undegraded":"2024-05-27T06:56:42.433330+0000","last_fullsized":"2024-05-27T06:56:42.433330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341388+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T06:56:40.341388+0000","last_peered":"2024-05-27T06:56:40.341388+0000","last_clean":"2024-05-27T06:56:40.341388+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T06:56:40.341388+0000","last_undegraded":"2024-05-27T06:56:40.341388+0000","last_fullsized":"2024-05-27T06:56:40.341388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349485+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T06:56:40.349485+0000","last_peered":"2024-05-27T06:56:40.349485+0000","last_clean":"2024-05-27T06:56:40.349485+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T06:56:40.349485+0000","last_undegraded":"2024-05-27T06:56:40.349485+0000","last_fullsized":"2024-05-27T06:56:40.349485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:20.571750+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T06:57:20.571750+0000","last_peered":"2024-05-27T06:57:20.571750+0000","last_clean":"2024-05-27T06:57:20.571750+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T06:57:20.571750+0000","last_undegraded":"2024-05-27T06:57:20.571750+0000","last_fullsized":"2024-05-27T06:57:20.571750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400902+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T06:56:40.400902+0000","last_peered":"2024-05-27T06:56:40.400902+0000","last_clean":"2024-05-27T06:56:40.400902+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T06:56:40.400902+0000","last_undegraded":"2024-05-27T06:56:40.400902+0000","last_fullsized":"2024-05-27T06:56:40.400902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349049+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T06:56:40.349049+0000","last_peered":"2024-05-27T06:56:40.349049+0000","last_clean":"2024-05-27T06:56:40.349049+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T06:56:40.349049+0000","last_undegraded":"2024-05-27T06:56:40.349049+0000","last_fullsized":"2024-05-27T06:56:40.349049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.421995+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T06:56:41.421995+0000","last_peered":"2024-05-27T06:56:41.421995+0000","last_clean":"2024-05-27T06:56:41.421995+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T06:56:41.421995+0000","last_undegraded":"2024-05-27T06:56:41.421995+0000","last_fullsized":"2024-05-27T06:56:41.421995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401736+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T06:56:40.401736+0000","last_peered":"2024-05-27T06:56:40.401736+0000","last_clean":"2024-05-27T06:56:40.401736+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T06:56:40.401736+0000","last_undegraded":"2024-05-27T06:56:40.401736+0000","last_fullsized":"2024-05-27T06:56:40.401736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408090+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T06:56:40.408090+0000","last_peered":"2024-05-27T06:56:40.408090+0000","last_clean":"2024-05-27T06:56:40.408090+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T06:56:40.408090+0000","last_undegraded":"2024-05-27T06:56:40.408090+0000","last_fullsized":"2024-05-27T06:56:40.408090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400695+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T06:56:40.400695+0000","last_peered":"2024-05-27T06:56:40.400695+0000","last_clean":"2024-05-27T06:56:40.400695+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T06:56:40.400695+0000","last_undegraded":"2024-05-27T06:56:40.400695+0000","last_fullsized":"2024-05-27T06:56:40.400695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404087+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T06:56:40.404087+0000","last_peered":"2024-05-27T06:56:40.404087+0000","last_clean":"2024-05-27T06:56:40.404087+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T06:56:40.404087+0000","last_undegraded":"2024-05-27T06:56:40.404087+0000","last_fullsized":"2024-05-27T06:56:40.404087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399464+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T06:56:40.399464+0000","last_peered":"2024-05-27T06:56:40.399464+0000","last_clean":"2024-05-27T06:56:40.399464+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T06:56:40.399464+0000","last_undegraded":"2024-05-27T06:56:40.399464+0000","last_fullsized":"2024-05-27T06:56:40.399464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434099+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T06:56:42.434099+0000","last_peered":"2024-05-27T06:56:42.434099+0000","last_clean":"2024-05-27T06:56:42.434099+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T06:56:42.434099+0000","last_undegraded":"2024-05-27T06:56:42.434099+0000","last_fullsized":"2024-05-27T06:56:42.434099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399235+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T06:56:40.399235+0000","last_peered":"2024-05-27T06:56:40.399235+0000","last_clean":"2024-05-27T06:56:40.399235+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T06:56:40.399235+0000","last_undegraded":"2024-05-27T06:56:40.399235+0000","last_fullsized":"2024-05-27T06:56:40.399235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402717+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T06:56:40.402717+0000","last_peered":"2024-05-27T06:56:40.402717+0000","last_clean":"2024-05-27T06:56:40.402717+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T06:56:40.402717+0000","last_undegraded":"2024-05-27T06:56:40.402717+0000","last_fullsized":"2024-05-27T06:56:40.402717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":126,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:22.576878+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T06:57:22.576878+0000","last_peered":"2024-05-27T06:57:22.576878+0000","last_clean":"2024-05-27T06:57:22.576878+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T06:57:22.576878+0000","last_undegraded":"2024-05-27T06:57:22.576878+0000","last_fullsized":"2024-05-27T06:57:22.576878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396731+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T06:56:40.396731+0000","last_peered":"2024-05-27T06:56:40.396731+0000","last_clean":"2024-05-27T06:56:40.396731+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T06:56:40.396731+0000","last_undegraded":"2024-05-27T06:56:40.396731+0000","last_fullsized":"2024-05-27T06:56:40.396731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398553+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T06:56:40.398553+0000","last_peered":"2024-05-27T06:56:40.398553+0000","last_clean":"2024-05-27T06:56:40.398553+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T06:56:40.398553+0000","last_undegraded":"2024-05-27T06:56:40.398553+0000","last_fullsized":"2024-05-27T06:56:40.398553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432920+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T06:56:42.432920+0000","last_peered":"2024-05-27T06:56:42.432920+0000","last_clean":"2024-05-27T06:56:42.432920+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T06:56:42.432920+0000","last_undegraded":"2024-05-27T06:56:42.432920+0000","last_fullsized":"2024-05-27T06:56:42.432920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399521+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T06:56:40.399521+0000","last_peered":"2024-05-27T06:56:40.399521+0000","last_clean":"2024-05-27T06:56:40.399521+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T06:56:40.399521+0000","last_undegraded":"2024-05-27T06:56:40.399521+0000","last_fullsized":"2024-05-27T06:56:40.399521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342345+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T06:56:40.342345+0000","last_peered":"2024-05-27T06:56:40.342345+0000","last_clean":"2024-05-27T06:56:40.342345+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T06:56:40.342345+0000","last_undegraded":"2024-05-27T06:56:40.342345+0000","last_fullsized":"2024-05-27T06:56:40.342345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422575+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T06:56:41.422575+0000","last_peered":"2024-05-27T06:56:41.422575+0000","last_clean":"2024-05-27T06:56:41.422575+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T06:56:41.422575+0000","last_undegraded":"2024-05-27T06:56:41.422575+0000","last_fullsized":"2024-05-27T06:56:41.422575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396556+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T06:56:40.396556+0000","last_peered":"2024-05-27T06:56:40.396556+0000","last_clean":"2024-05-27T06:56:40.396556+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T06:56:40.396556+0000","last_undegraded":"2024-05-27T06:56:40.396556+0000","last_fullsized":"2024-05-27T06:56:40.396556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399765+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T06:56:40.399765+0000","last_peered":"2024-05-27T06:56:40.399765+0000","last_clean":"2024-05-27T06:56:40.399765+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T06:56:40.399765+0000","last_undegraded":"2024-05-27T06:56:40.399765+0000","last_fullsized":"2024-05-27T06:56:40.399765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":185,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:22.598560+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T06:57:22.598560+0000","last_peered":"2024-05-27T06:57:22.598560+0000","last_clean":"2024-05-27T06:57:22.598560+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T06:57:22.598560+0000","last_undegraded":"2024-05-27T06:57:22.598560+0000","last_fullsized":"2024-05-27T06:57:22.598560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66481,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":109,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404127+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T06:56:40.404127+0000","last_peered":"2024-05-27T06:56:40.404127+0000","last_clean":"2024-05-27T06:56:40.404127+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T06:56:40.404127+0000","last_undegraded":"2024-05-27T06:56:40.404127+0000","last_fullsized":"2024-05-27T06:56:40.404127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342124+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T06:56:40.342124+0000","last_peered":"2024-05-27T06:56:40.342124+0000","last_clean":"2024-05-27T06:56:40.342124+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T06:56:40.342124+0000","last_undegraded":"2024-05-27T06:56:40.342124+0000","last_fullsized":"2024-05-27T06:56:40.342124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449381+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T06:56:44.449381+0000","last_peered":"2024-05-27T06:56:44.449381+0000","last_clean":"2024-05-27T06:56:44.449381+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T06:56:44.449381+0000","last_undegraded":"2024-05-27T06:56:44.449381+0000","last_fullsized":"2024-05-27T06:56:44.449381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335193+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T06:56:40.335193+0000","last_peered":"2024-05-27T06:56:40.335193+0000","last_clean":"2024-05-27T06:56:40.335193+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T06:56:40.335193+0000","last_undegraded":"2024-05-27T06:56:40.335193+0000","last_fullsized":"2024-05-27T06:56:40.335193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401346+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T06:56:40.401346+0000","last_peered":"2024-05-27T06:56:40.401346+0000","last_clean":"2024-05-27T06:56:40.401346+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T06:56:40.401346+0000","last_undegraded":"2024-05-27T06:56:40.401346+0000","last_fullsized":"2024-05-27T06:56:40.401346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434087+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T06:56:42.434087+0000","last_peered":"2024-05-27T06:56:42.434087+0000","last_clean":"2024-05-27T06:56:42.434087+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T06:56:42.434087+0000","last_undegraded":"2024-05-27T06:56:42.434087+0000","last_fullsized":"2024-05-27T06:56:42.434087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338964+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T06:56:40.338964+0000","last_peered":"2024-05-27T06:56:40.338964+0000","last_clean":"2024-05-27T06:56:40.338964+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T06:56:40.338964+0000","last_undegraded":"2024-05-27T06:56:40.338964+0000","last_fullsized":"2024-05-27T06:56:40.338964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402073+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T06:56:40.402073+0000","last_peered":"2024-05-27T06:56:40.402073+0000","last_clean":"2024-05-27T06:56:40.402073+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T06:56:40.402073+0000","last_undegraded":"2024-05-27T06:56:40.402073+0000","last_fullsized":"2024-05-27T06:56:40.402073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:20.577249+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T06:57:20.577249+0000","last_peered":"2024-05-27T06:57:20.577249+0000","last_clean":"2024-05-27T06:57:20.577249+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T06:57:20.577249+0000","last_undegraded":"2024-05-27T06:57:20.577249+0000","last_fullsized":"2024-05-27T06:57:20.577249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338007+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T06:56:40.338007+0000","last_peered":"2024-05-27T06:56:40.338007+0000","last_clean":"2024-05-27T06:56:40.338007+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T06:56:40.338007+0000","last_undegraded":"2024-05-27T06:56:40.338007+0000","last_fullsized":"2024-05-27T06:56:40.338007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335370+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T06:56:40.335370+0000","last_peered":"2024-05-27T06:56:40.335370+0000","last_clean":"2024-05-27T06:56:40.335370+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T06:56:40.335370+0000","last_undegraded":"2024-05-27T06:56:40.335370+0000","last_fullsized":"2024-05-27T06:56:40.335370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449699+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T06:56:44.449699+0000","last_peered":"2024-05-27T06:56:44.449699+0000","last_clean":"2024-05-27T06:56:44.449699+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T06:56:44.449699+0000","last_undegraded":"2024-05-27T06:56:44.449699+0000","last_fullsized":"2024-05-27T06:56:44.449699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340408+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T06:56:40.340408+0000","last_peered":"2024-05-27T06:56:40.340408+0000","last_clean":"2024-05-27T06:56:40.340408+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T06:56:40.340408+0000","last_undegraded":"2024-05-27T06:56:40.340408+0000","last_fullsized":"2024-05-27T06:56:40.340408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340047+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T06:56:40.340047+0000","last_peered":"2024-05-27T06:56:40.340047+0000","last_clean":"2024-05-27T06:56:40.340047+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T06:56:40.340047+0000","last_undegraded":"2024-05-27T06:56:40.340047+0000","last_fullsized":"2024-05-27T06:56:40.340047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451292+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T06:56:44.451292+0000","last_peered":"2024-05-27T06:56:44.451292+0000","last_clean":"2024-05-27T06:56:44.451292+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T06:56:44.451292+0000","last_undegraded":"2024-05-27T06:56:44.451292+0000","last_fullsized":"2024-05-27T06:56:44.451292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408039+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T06:56:40.408039+0000","last_peered":"2024-05-27T06:56:40.408039+0000","last_clean":"2024-05-27T06:56:40.408039+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T06:56:40.408039+0000","last_undegraded":"2024-05-27T06:56:40.408039+0000","last_fullsized":"2024-05-27T06:56:40.408039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401367+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T06:56:40.401367+0000","last_peered":"2024-05-27T06:56:40.401367+0000","last_clean":"2024-05-27T06:56:40.401367+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T06:56:40.401367+0000","last_undegraded":"2024-05-27T06:56:40.401367+0000","last_fullsized":"2024-05-27T06:56:40.401367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422923+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T06:56:41.422923+0000","last_peered":"2024-05-27T06:56:41.422923+0000","last_clean":"2024-05-27T06:56:41.422923+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T06:56:41.422923+0000","last_undegraded":"2024-05-27T06:56:41.422923+0000","last_fullsized":"2024-05-27T06:56:41.422923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394931+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T06:56:40.394931+0000","last_peered":"2024-05-27T06:56:40.394931+0000","last_clean":"2024-05-27T06:56:40.394931+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T06:56:40.394931+0000","last_undegraded":"2024-05-27T06:56:40.394931+0000","last_fullsized":"2024-05-27T06:56:40.394931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335765+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T06:56:40.335765+0000","last_peered":"2024-05-27T06:56:40.335765+0000","last_clean":"2024-05-27T06:56:40.335765+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T06:56:40.335765+0000","last_undegraded":"2024-05-27T06:56:40.335765+0000","last_fullsized":"2024-05-27T06:56:40.335765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:23.577486+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T06:57:23.577486+0000","last_peered":"2024-05-27T06:57:23.577486+0000","last_clean":"2024-05-27T06:57:23.577486+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T06:57:23.577486+0000","last_undegraded":"2024-05-27T06:57:23.577486+0000","last_fullsized":"2024-05-27T06:57:23.577486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.393918+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T06:56:40.393918+0000","last_peered":"2024-05-27T06:56:40.393918+0000","last_clean":"2024-05-27T06:56:40.393918+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T06:56:40.393918+0000","last_undegraded":"2024-05-27T06:56:40.393918+0000","last_fullsized":"2024-05-27T06:56:40.393918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335014+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T06:56:40.335014+0000","last_peered":"2024-05-27T06:56:40.335014+0000","last_clean":"2024-05-27T06:56:40.335014+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T06:56:40.335014+0000","last_undegraded":"2024-05-27T06:56:40.335014+0000","last_fullsized":"2024-05-27T06:56:40.335014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340613+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T06:56:40.340613+0000","last_peered":"2024-05-27T06:56:40.340613+0000","last_clean":"2024-05-27T06:56:40.340613+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T06:56:40.340613+0000","last_undegraded":"2024-05-27T06:56:40.340613+0000","last_fullsized":"2024-05-27T06:56:40.340613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:20.575797+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T06:57:20.575797+0000","last_peered":"2024-05-27T06:57:20.575797+0000","last_clean":"2024-05-27T06:57:20.575797+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T06:57:20.575797+0000","last_undegraded":"2024-05-27T06:57:20.575797+0000","last_fullsized":"2024-05-27T06:57:20.575797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339695+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T06:56:40.339695+0000","last_peered":"2024-05-27T06:56:40.339695+0000","last_clean":"2024-05-27T06:56:40.339695+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T06:56:40.339695+0000","last_undegraded":"2024-05-27T06:56:40.339695+0000","last_fullsized":"2024-05-27T06:56:40.339695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.405949+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T06:56:40.405949+0000","last_peered":"2024-05-27T06:56:40.405949+0000","last_clean":"2024-05-27T06:56:40.405949+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T06:56:40.405949+0000","last_undegraded":"2024-05-27T06:56:40.405949+0000","last_fullsized":"2024-05-27T06:56:40.405949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450805+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T06:56:44.450805+0000","last_peered":"2024-05-27T06:56:44.450805+0000","last_clean":"2024-05-27T06:56:44.450805+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T06:56:44.450805+0000","last_undegraded":"2024-05-27T06:56:44.450805+0000","last_fullsized":"2024-05-27T06:56:44.450805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.337815+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T06:56:40.337815+0000","last_peered":"2024-05-27T06:56:40.337815+0000","last_clean":"2024-05-27T06:56:40.337815+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T06:56:40.337815+0000","last_undegraded":"2024-05-27T06:56:40.337815+0000","last_fullsized":"2024-05-27T06:56:40.337815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342055+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T06:56:40.342055+0000","last_peered":"2024-05-27T06:56:40.342055+0000","last_clean":"2024-05-27T06:56:40.342055+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T06:56:40.342055+0000","last_undegraded":"2024-05-27T06:56:40.342055+0000","last_fullsized":"2024-05-27T06:56:40.342055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.334716+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T06:56:40.334716+0000","last_peered":"2024-05-27T06:56:40.334716+0000","last_clean":"2024-05-27T06:56:40.334716+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T06:56:40.334716+0000","last_undegraded":"2024-05-27T06:56:40.334716+0000","last_fullsized":"2024-05-27T06:56:40.334716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432537+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T06:56:42.432537+0000","last_peered":"2024-05-27T06:56:42.432537+0000","last_clean":"2024-05-27T06:56:42.432537+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T06:56:42.432537+0000","last_undegraded":"2024-05-27T06:56:42.432537+0000","last_fullsized":"2024-05-27T06:56:42.432537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402420+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T06:56:40.402420+0000","last_peered":"2024-05-27T06:56:40.402420+0000","last_clean":"2024-05-27T06:56:40.402420+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T06:56:40.402420+0000","last_undegraded":"2024-05-27T06:56:40.402420+0000","last_fullsized":"2024-05-27T06:56:40.402420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450385+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T06:56:44.450385+0000","last_peered":"2024-05-27T06:56:44.450385+0000","last_clean":"2024-05-27T06:56:44.450385+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T06:56:44.450385+0000","last_undegraded":"2024-05-27T06:56:44.450385+0000","last_fullsized":"2024-05-27T06:56:44.450385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335904+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T06:56:40.335904+0000","last_peered":"2024-05-27T06:56:40.335904+0000","last_clean":"2024-05-27T06:56:40.335904+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T06:56:40.335904+0000","last_undegraded":"2024-05-27T06:56:40.335904+0000","last_fullsized":"2024-05-27T06:56:40.335904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.403943+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T06:56:40.403943+0000","last_peered":"2024-05-27T06:56:40.403943+0000","last_clean":"2024-05-27T06:56:40.403943+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T06:56:40.403943+0000","last_undegraded":"2024-05-27T06:56:40.403943+0000","last_fullsized":"2024-05-27T06:56:40.403943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:16.307934+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T06:57:16.307934+0000","last_peered":"2024-05-27T06:57:16.307934+0000","last_clean":"2024-05-27T06:57:16.307934+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T06:57:16.307934+0000","last_undegraded":"2024-05-27T06:57:16.307934+0000","last_fullsized":"2024-05-27T06:57:16.307934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402232+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T06:56:40.402232+0000","last_peered":"2024-05-27T06:56:40.402232+0000","last_clean":"2024-05-27T06:56:40.402232+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T06:56:40.402232+0000","last_undegraded":"2024-05-27T06:56:40.402232+0000","last_fullsized":"2024-05-27T06:56:40.402232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348176+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T06:56:40.348176+0000","last_peered":"2024-05-27T06:56:40.348176+0000","last_clean":"2024-05-27T06:56:40.348176+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T06:56:40.348176+0000","last_undegraded":"2024-05-27T06:56:40.348176+0000","last_fullsized":"2024-05-27T06:56:40.348176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449227+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T06:56:44.449227+0000","last_peered":"2024-05-27T06:56:44.449227+0000","last_clean":"2024-05-27T06:56:44.449227+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T06:56:44.449227+0000","last_undegraded":"2024-05-27T06:56:44.449227+0000","last_fullsized":"2024-05-27T06:56:44.449227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399744+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T06:56:40.399744+0000","last_peered":"2024-05-27T06:56:40.399744+0000","last_clean":"2024-05-27T06:56:40.399744+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T06:56:40.399744+0000","last_undegraded":"2024-05-27T06:56:40.399744+0000","last_fullsized":"2024-05-27T06:56:40.399744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344023+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T06:56:40.344023+0000","last_peered":"2024-05-27T06:56:40.344023+0000","last_clean":"2024-05-27T06:56:40.344023+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T06:56:40.344023+0000","last_undegraded":"2024-05-27T06:56:40.344023+0000","last_fullsized":"2024-05-27T06:56:40.344023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394827+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T06:56:40.394827+0000","last_peered":"2024-05-27T06:56:40.394827+0000","last_clean":"2024-05-27T06:56:40.394827+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T06:56:40.394827+0000","last_undegraded":"2024-05-27T06:56:40.394827+0000","last_fullsized":"2024-05-27T06:56:40.394827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400252+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T06:56:40.400252+0000","last_peered":"2024-05-27T06:56:40.400252+0000","last_clean":"2024-05-27T06:56:40.400252+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T06:56:40.400252+0000","last_undegraded":"2024-05-27T06:56:40.400252+0000","last_fullsized":"2024-05-27T06:56:40.400252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402480+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T06:56:40.402480+0000","last_peered":"2024-05-27T06:56:40.402480+0000","last_clean":"2024-05-27T06:56:40.402480+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T06:56:40.402480+0000","last_undegraded":"2024-05-27T06:56:40.402480+0000","last_fullsized":"2024-05-27T06:56:40.402480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434757+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T06:56:42.434757+0000","last_peered":"2024-05-27T06:56:42.434757+0000","last_clean":"2024-05-27T06:56:42.434757+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T06:56:42.434757+0000","last_undegraded":"2024-05-27T06:56:42.434757+0000","last_fullsized":"2024-05-27T06:56:42.434757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399660+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T06:56:40.399660+0000","last_peered":"2024-05-27T06:56:40.399660+0000","last_clean":"2024-05-27T06:56:40.399660+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T06:56:40.399660+0000","last_undegraded":"2024-05-27T06:56:40.399660+0000","last_fullsized":"2024-05-27T06:56:40.399660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.343231+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T06:56:40.343231+0000","last_peered":"2024-05-27T06:56:40.343231+0000","last_clean":"2024-05-27T06:56:40.343231+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T06:56:40.343231+0000","last_undegraded":"2024-05-27T06:56:40.343231+0000","last_fullsized":"2024-05-27T06:56:40.343231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.407953+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T06:56:42.407953+0000","last_peered":"2024-05-27T06:56:42.407953+0000","last_clean":"2024-05-27T06:56:42.407953+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T06:56:42.407953+0000","last_undegraded":"2024-05-27T06:56:42.407953+0000","last_fullsized":"2024-05-27T06:56:42.407953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.403915+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T06:56:40.403915+0000","last_peered":"2024-05-27T06:56:40.403915+0000","last_clean":"2024-05-27T06:56:40.403915+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T06:56:40.403915+0000","last_undegraded":"2024-05-27T06:56:40.403915+0000","last_fullsized":"2024-05-27T06:56:40.403915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.343929+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T06:56:40.343929+0000","last_peered":"2024-05-27T06:56:40.343929+0000","last_clean":"2024-05-27T06:56:40.343929+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T06:56:40.343929+0000","last_undegraded":"2024-05-27T06:56:40.343929+0000","last_fullsized":"2024-05-27T06:56:40.343929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451041+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T06:56:44.451041+0000","last_peered":"2024-05-27T06:56:44.451041+0000","last_clean":"2024-05-27T06:56:44.451041+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T06:56:44.451041+0000","last_undegraded":"2024-05-27T06:56:44.451041+0000","last_fullsized":"2024-05-27T06:56:44.451041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.338725+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T06:56:40.338725+0000","last_peered":"2024-05-27T06:56:40.338725+0000","last_clean":"2024-05-27T06:56:40.338725+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T06:56:40.338725+0000","last_undegraded":"2024-05-27T06:56:40.338725+0000","last_fullsized":"2024-05-27T06:56:40.338725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396142+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T06:56:40.396142+0000","last_peered":"2024-05-27T06:56:40.396142+0000","last_clean":"2024-05-27T06:56:40.396142+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T06:56:40.396142+0000","last_undegraded":"2024-05-27T06:56:40.396142+0000","last_fullsized":"2024-05-27T06:56:40.396142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:23.583327+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T06:57:23.583327+0000","last_peered":"2024-05-27T06:57:23.583327+0000","last_clean":"2024-05-27T06:57:23.583327+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T06:57:23.583327+0000","last_undegraded":"2024-05-27T06:57:23.583327+0000","last_fullsized":"2024-05-27T06:57:23.583327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340608+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T06:56:40.340608+0000","last_peered":"2024-05-27T06:56:40.340608+0000","last_clean":"2024-05-27T06:56:40.340608+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T06:56:40.340608+0000","last_undegraded":"2024-05-27T06:56:40.340608+0000","last_fullsized":"2024-05-27T06:56:40.340608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398710+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T06:56:40.398710+0000","last_peered":"2024-05-27T06:56:40.398710+0000","last_clean":"2024-05-27T06:56:40.398710+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T06:56:40.398710+0000","last_undegraded":"2024-05-27T06:56:40.398710+0000","last_fullsized":"2024-05-27T06:56:40.398710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451295+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T06:56:44.451295+0000","last_peered":"2024-05-27T06:56:44.451295+0000","last_clean":"2024-05-27T06:56:44.451295+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T06:56:44.451295+0000","last_undegraded":"2024-05-27T06:56:44.451295+0000","last_fullsized":"2024-05-27T06:56:44.451295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339489+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T06:56:40.339489+0000","last_peered":"2024-05-27T06:56:40.339489+0000","last_clean":"2024-05-27T06:56:40.339489+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T06:56:40.339489+0000","last_undegraded":"2024-05-27T06:56:40.339489+0000","last_fullsized":"2024-05-27T06:56:40.339489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339644+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T06:56:40.339644+0000","last_peered":"2024-05-27T06:56:40.339644+0000","last_clean":"2024-05-27T06:56:40.339644+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T06:56:40.339644+0000","last_undegraded":"2024-05-27T06:56:40.339644+0000","last_fullsized":"2024-05-27T06:56:40.339644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432981+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T06:56:42.432981+0000","last_peered":"2024-05-27T06:56:42.432981+0000","last_clean":"2024-05-27T06:56:42.432981+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T06:56:42.432981+0000","last_undegraded":"2024-05-27T06:56:42.432981+0000","last_fullsized":"2024-05-27T06:56:42.432981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340532+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T06:56:40.340532+0000","last_peered":"2024-05-27T06:56:40.340532+0000","last_clean":"2024-05-27T06:56:40.340532+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T06:56:40.340532+0000","last_undegraded":"2024-05-27T06:56:40.340532+0000","last_fullsized":"2024-05-27T06:56:40.340532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.400049+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T06:56:40.400049+0000","last_peered":"2024-05-27T06:56:40.400049+0000","last_clean":"2024-05-27T06:56:40.400049+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T06:56:40.400049+0000","last_undegraded":"2024-05-27T06:56:40.400049+0000","last_fullsized":"2024-05-27T06:56:40.400049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348904+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T06:56:40.348904+0000","last_peered":"2024-05-27T06:56:40.348904+0000","last_clean":"2024-05-27T06:56:40.348904+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T06:56:40.348904+0000","last_undegraded":"2024-05-27T06:56:40.348904+0000","last_fullsized":"2024-05-27T06:56:40.348904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349077+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T06:56:40.349077+0000","last_peered":"2024-05-27T06:56:40.349077+0000","last_clean":"2024-05-27T06:56:40.349077+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T06:56:40.349077+0000","last_undegraded":"2024-05-27T06:56:40.349077+0000","last_fullsized":"2024-05-27T06:56:40.349077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399697+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T06:56:40.399697+0000","last_peered":"2024-05-27T06:56:40.399697+0000","last_clean":"2024-05-27T06:56:40.399697+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T06:56:40.399697+0000","last_undegraded":"2024-05-27T06:56:40.399697+0000","last_fullsized":"2024-05-27T06:56:40.399697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.336012+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T06:56:40.336012+0000","last_peered":"2024-05-27T06:56:40.336012+0000","last_clean":"2024-05-27T06:56:40.336012+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T06:56:40.336012+0000","last_undegraded":"2024-05-27T06:56:40.336012+0000","last_fullsized":"2024-05-27T06:56:40.336012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.405964+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T06:56:40.405964+0000","last_peered":"2024-05-27T06:56:40.405964+0000","last_clean":"2024-05-27T06:56:40.405964+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T06:56:40.405964+0000","last_undegraded":"2024-05-27T06:56:40.405964+0000","last_fullsized":"2024-05-27T06:56:40.405964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.350143+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T06:56:40.350143+0000","last_peered":"2024-05-27T06:56:40.350143+0000","last_clean":"2024-05-27T06:56:40.350143+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T06:56:40.350143+0000","last_undegraded":"2024-05-27T06:56:40.350143+0000","last_fullsized":"2024-05-27T06:56:40.350143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402572+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T06:56:40.402572+0000","last_peered":"2024-05-27T06:56:40.402572+0000","last_clean":"2024-05-27T06:56:40.402572+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T06:56:40.402572+0000","last_undegraded":"2024-05-27T06:56:40.402572+0000","last_fullsized":"2024-05-27T06:56:40.402572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.334548+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T06:56:40.334548+0000","last_peered":"2024-05-27T06:56:40.334548+0000","last_clean":"2024-05-27T06:56:40.334548+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T06:56:40.334548+0000","last_undegraded":"2024-05-27T06:56:40.334548+0000","last_fullsized":"2024-05-27T06:56:40.334548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402965+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T06:56:40.402965+0000","last_peered":"2024-05-27T06:56:40.402965+0000","last_clean":"2024-05-27T06:56:40.402965+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T06:56:40.402965+0000","last_undegraded":"2024-05-27T06:56:40.402965+0000","last_fullsized":"2024-05-27T06:56:40.402965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396995+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T06:56:40.396995+0000","last_peered":"2024-05-27T06:56:40.396995+0000","last_clean":"2024-05-27T06:56:40.396995+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T06:56:40.396995+0000","last_undegraded":"2024-05-27T06:56:40.396995+0000","last_fullsized":"2024-05-27T06:56:40.396995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335453+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T06:56:40.335453+0000","last_peered":"2024-05-27T06:56:40.335453+0000","last_clean":"2024-05-27T06:56:40.335453+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T06:56:40.335453+0000","last_undegraded":"2024-05-27T06:56:40.335453+0000","last_fullsized":"2024-05-27T06:56:40.335453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402340+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T06:56:40.402340+0000","last_peered":"2024-05-27T06:56:40.402340+0000","last_clean":"2024-05-27T06:56:40.402340+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T06:56:40.402340+0000","last_undegraded":"2024-05-27T06:56:40.402340+0000","last_fullsized":"2024-05-27T06:56:40.402340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74076,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":432,"num_read_kb":295,"num_write":252,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":222228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678064,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874294,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1174757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234053,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":521377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430286,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1662976,"data_stored":1118265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":516894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":584405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":518350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986275,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":587449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822748,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T06:57:28.537 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T06:57:28.538 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T06:57:28.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T06:57:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:29.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:29 smithi016 ceph-mon[31415]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:29.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:29 smithi016 ceph-mon[31415]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T06:57:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:29 smithi164 ceph-mon[35327]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:29 smithi164 ceph-mon[35327]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T06:57:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:29 smithi069 ceph-mon[35276]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:57:29.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:29 smithi069 ceph-mon[35276]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T06:57:31.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:31 smithi016 ceph-mon[31415]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:31 smithi164 ceph-mon[35327]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:31.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:31 smithi069 ceph-mon[35276]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.683+0000 7f4c9de00700 1 -- 172.21.15.16:0/2802243646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98104350 msgr2=0x7f4c98104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.683+0000 7f4c9de00700 1 --2- 172.21.15.16:0/2802243646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98104350 0x7f4c98104760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009990 tx=0x7f4c84009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 -- 172.21.15.16:0/2802243646 shutdown_connections 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 --2- 172.21.15.16:0/2802243646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 0x7f4c9810b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 --2- 172.21.15.16:0/2802243646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98103220 0x7f4c98103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 --2- 172.21.15.16:0/2802243646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98104350 0x7f4c98104760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 -- 172.21.15.16:0/2802243646 >> 172.21.15.16:0/2802243646 conn(0x7f4c980fe860 msgr2=0x7f4c98100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 -- 172.21.15.16:0/2802243646 shutdown_connections 2024-05-27T06:57:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.684+0000 7f4c9de00700 1 -- 172.21.15.16:0/2802243646 wait complete. 2024-05-27T06:57:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.685+0000 7f4c9de00700 1 Processor -- start 2024-05-27T06:57:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.685+0000 7f4c9de00700 1 -- start start 2024-05-27T06:57:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98104350 0x7f4c9819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 0x7f4c981a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810eb80 con 0x7f4c98103220 2024-05-27T06:57:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810ea00 con 0x7f4c98104cf0 2024-05-27T06:57:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c9de00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810ed00 con 0x7f4c98104350 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c96a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c96a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55296/0 (socket says 172.21.15.16:55296) 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.686+0000 7f4c96a00700 1 -- 172.21.15.16:0/1584840079 learned_addr learned my addr 172.21.15.16:0/1584840079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96000700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98104350 0x7f4c9819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c97400700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 0x7f4c981a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 -- 172.21.15.16:0/1584840079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98104350 msgr2=0x7f4c9819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98104350 0x7f4c9819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 -- 172.21.15.16:0/1584840079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 msgr2=0x7f4c981a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 0x7f4c981a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 -- 172.21.15.16:0/1584840079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c981a4630 con 0x7f4c98103220 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.687+0000 7f4c96a00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e940 tx=0x7f4c8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.688+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800cbd0 con 0x7f4c98103220 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.688+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c8800ee90 con 0x7f4c98103220 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.688+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c981a4920 con 0x7f4c98103220 2024-05-27T06:57:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.688+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88003d00 con 0x7f4c98103220 2024-05-27T06:57:31.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.688+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c981a4eb0 con 0x7f4c98103220 2024-05-27T06:57:31.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.690+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c980512b0 con 0x7f4c98103220 2024-05-27T06:57:31.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.692+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4c88003ea0 con 0x7f4c98103220 2024-05-27T06:57:31.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.694+0000 7f4c9ca00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 0x7f4c7c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:31.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.694+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4c880e2a90 con 0x7f4c98103220 2024-05-27T06:57:31.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.694+0000 7f4c96000700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 0x7f4c7c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:31.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.695+0000 7f4c96000700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 0x7f4c7c0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b650 tx=0x7f4c8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:31.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:31.697+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c880aa2d0 con 0x7f4c98103220 2024-05-27T06:57:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.032+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4c981a5140 con 0x7f4c98103220 2024-05-27T06:57:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.036+0000 7f4c9ca00700 1 -- 172.21.15.16:0/1584840079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7f4c880a9bd0 con 0x7f4c98103220 2024-05-27T06:57:32.029 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-05-27T06:57:32.029 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDcLlRm73kNAhAAU1CY5OCPAosZNmndC3+Ghg== 2024-05-27T06:57:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 msgr2=0x7f4c7c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 0x7f4c7c0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b650 tx=0x7f4c8401a040 comp rx=0 tx=0).stop 2024-05-27T06:57:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 msgr2=0x7f4c9819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e940 tx=0x7f4c8800ec50 comp rx=0 tx=0).stop 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 shutdown_connections 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f4c7c0b2990 0x7f4c7c0b4ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c98104cf0 0x7f4c981a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c98103220 0x7f4c9819cad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 --2- 172.21.15.16:0/1584840079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c98104350 0x7f4c9819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.041+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 >> 172.21.15.16:0/1584840079 conn(0x7f4c980fe860 msgr2=0x7f4c98100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.042+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 shutdown_connections 2024-05-27T06:57:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:32.042+0000 7f4c9de00700 1 -- 172.21.15.16:0/1584840079 wait complete. 2024-05-27T06:57:32.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:32 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T06:57:32.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:32 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T06:57:32.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:32 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T06:57:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:32 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T06:57:32.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:32 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T06:57:32.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:32 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1584840079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T06:57:32.892 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T06:57:32.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T06:57:32.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T06:57:33.004 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T06:57:33.004 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T06:57:33.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T06:57:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:33 smithi164 ceph-mon[35327]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:33.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:33 smithi016 ceph-mon[31415]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:33.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:33 smithi069 ceph-mon[35276]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-27T06:57:35.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:35 smithi016 ceph-mon[31415]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:57:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:35 smithi164 ceph-mon[35327]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:57:35.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:35 smithi069 ceph-mon[35276]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 -- 172.21.15.16:0/1214344564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d581039c0 msgr2=0x7f2d581106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 --2- 172.21.15.16:0/1214344564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d581039c0 0x7f2d581106e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009990 tx=0x7f2d44009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 -- 172.21.15.16:0/1214344564 shutdown_connections 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 --2- 172.21.15.16:0/1214344564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58110c20 0x7f2d58113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 --2- 172.21.15.16:0/1214344564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d58103070 0x7f2d58103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 --2- 172.21.15.16:0/1214344564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d581039c0 0x7f2d581106e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 -- 172.21.15.16:0/1214344564 >> 172.21.15.16:0/1214344564 conn(0x7f2d580fede0 msgr2=0x7f2d58101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 -- 172.21.15.16:0/1214344564 shutdown_connections 2024-05-27T06:57:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.994+0000 7f2d5da00700 1 -- 172.21.15.16:0/1214344564 wait complete. 2024-05-27T06:57:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.995+0000 7f2d5da00700 1 Processor -- start 2024-05-27T06:57:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.995+0000 7f2d5da00700 1 -- start start 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58103070 0x7f2d5819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 0x7f2d5819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58116550 con 0x7f2d58110c20 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581163d0 con 0x7f2d581039c0 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d5da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581166d0 con 0x7f2d58103070 2024-05-27T06:57:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.996+0000 7f2d57400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47102/0 (socket says 172.21.15.16:47102) 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 -- 172.21.15.16:0/365123336 learned_addr learned my addr 172.21.15.16:0/365123336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d56a00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58103070 0x7f2d5819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d56000700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 0x7f2d5819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 -- 172.21.15.16:0/365123336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58103070 msgr2=0x7f2d5819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58103070 0x7f2d5819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 -- 172.21.15.16:0/365123336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 msgr2=0x7f2d5819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 0x7f2d5819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 -- 172.21.15.16:0/365123336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d581a1e50 con 0x7f2d58110c20 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d56000700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 0x7f2d5819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:57:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.997+0000 7f2d57400700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007ee0 tx=0x7f2d5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.998+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000dbd0 con 0x7f2d58110c20 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.998+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5000dd70 con 0x7f2d58110c20 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.998+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500058d0 con 0x7f2d58110c20 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.998+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d581a2140 con 0x7f2d58110c20 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.998+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d581a9a60 con 0x7f2d58110c20 2024-05-27T06:57:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:35.999+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d580512b0 con 0x7f2d58110c20 2024-05-27T06:57:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.002+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2d5001a020 con 0x7f2d58110c20 2024-05-27T06:57:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.003+0000 7f2d4f400700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 0x7f2d3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.003+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2d500e1e20 con 0x7f2d58110c20 2024-05-27T06:57:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.004+0000 7f2d56a00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 0x7f2d3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.004+0000 7f2d56a00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 0x7f2d3c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005950 tx=0x7f2d4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.008+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d50010070 con 0x7f2d58110c20 2024-05-27T06:57:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.310+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2d5802e5e0 con 0x7f2d58110c20 2024-05-27T06:57:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.328+0000 7f2d4f400700 1 -- 172.21.15.16:0/365123336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7f2d500a9630 con 0x7f2d58110c20 2024-05-27T06:57:36.322 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 msgr2=0x7f2d3c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 0x7f2d3c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005950 tx=0x7f2d4800b410 comp rx=0 tx=0).stop 2024-05-27T06:57:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 msgr2=0x7f2d581a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007ee0 tx=0x7f2d5000d3b0 comp rx=0 tx=0).stop 2024-05-27T06:57:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 shutdown_connections 2024-05-27T06:57:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2d3c0b2990 0x7f2d3c0b4ea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d58110c20 0x7f2d581a17b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d58103070 0x7f2d5819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 --2- 172.21.15.16:0/365123336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d581039c0 0x7f2d5819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.335+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 >> 172.21.15.16:0/365123336 conn(0x7f2d580fede0 msgr2=0x7f2d581128a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.336+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 shutdown_connections 2024-05-27T06:57:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:36.336+0000 7f2d5da00700 1 -- 172.21.15.16:0/365123336 wait complete. 2024-05-27T06:57:37.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2755801148},{"type":"v1","addr":"172.21.15.16:6801","nonce":2755801148}]},"active_addr":"172.21.15.16:6801/2755801148","active_change":"2024-05-27T06:44:37.876900+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34106,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":454643589}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3317326035}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":702672487}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3906409731}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":932115183}]}]} 2024-05-27T06:57:37.224 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T06:57:37.224 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T06:57:37.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:57:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:37 smithi164 ceph-mon[35327]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:37 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/365123336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T06:57:37.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:37 smithi016 ceph-mon[31415]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:37.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:37 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/365123336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T06:57:37.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:37 smithi069 ceph-mon[35276]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:37.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:37 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/365123336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T06:57:37.723 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:39.561 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:39 smithi016 ceph-mon[31415]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:39 smithi164 ceph-mon[35327]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:39.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:39 smithi069 ceph-mon[35276]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.171+0000 7fea78a00700 1 -- 172.21.15.16:0/1294271143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 msgr2=0x7fea74113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.171+0000 7fea78a00700 1 --2- 172.21.15.16:0/1294271143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea74113010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00b0a0 tx=0x7fea6c00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 -- 172.21.15.16:0/1294271143 shutdown_connections 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 --2- 172.21.15.16:0/1294271143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea74113010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 --2- 172.21.15.16:0/1294271143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74075af0 0x7fea74075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 --2- 172.21.15.16:0/1294271143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74076440 0x7fea741106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 -- 172.21.15.16:0/1294271143 >> 172.21.15.16:0/1294271143 conn(0x7fea74100030 msgr2=0x7fea74102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 -- 172.21.15.16:0/1294271143 shutdown_connections 2024-05-27T06:57:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.172+0000 7fea78a00700 1 -- 172.21.15.16:0/1294271143 wait complete. 2024-05-27T06:57:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.173+0000 7fea78a00700 1 Processor -- start 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.173+0000 7fea78a00700 1 -- start start 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74075af0 0x7fea7419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74076440 0x7fea7419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea741165d0 con 0x7fea74110bf0 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea74116450 con 0x7fea74076440 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea78a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea74116750 con 0x7fea74075af0 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea72800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea72800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47124/0 (socket says 172.21.15.16:47124) 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea72800700 1 -- 172.21.15.16:0/2409041137 learned_addr learned my addr 172.21.15.16:0/2409041137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea71e00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74075af0 0x7fea7419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.174+0000 7fea71400700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74076440 0x7fea7419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 -- 172.21.15.16:0/2409041137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74075af0 msgr2=0x7fea7419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74075af0 0x7fea7419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 -- 172.21.15.16:0/2409041137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74076440 msgr2=0x7fea7419d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74076440 0x7fea7419d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 -- 172.21.15.16:0/2409041137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea741a1e10 con 0x7fea74110bf0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea72800700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006070 tx=0x7fea6c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c021070 con 0x7fea74110bf0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea6c007b80 con 0x7fea74110bf0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea741a20a0 con 0x7fea74110bf0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.175+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0047a0 con 0x7fea74110bf0 2024-05-27T06:57:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.176+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a2580 con 0x7fea74110bf0 2024-05-27T06:57:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.177+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea740512b0 con 0x7fea74110bf0 2024-05-27T06:57:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.180+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fea6c003a00 con 0x7fea74110bf0 2024-05-27T06:57:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.182+0000 7fea6aa00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 0x7fea580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.182+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fea6c00f020 con 0x7fea74110bf0 2024-05-27T06:57:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.183+0000 7fea71e00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 0x7fea580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.183+0000 7fea71e00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 0x7fea580b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea64005950 tx=0x7fea6400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.186+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea6c02a020 con 0x7fea74110bf0 2024-05-27T06:57:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.422+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea7402e5e0 con 0x7fea74110bf0 2024-05-27T06:57:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.425+0000 7fea6aa00700 1 -- 172.21.15.16:0/2409041137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24708 (secure 0 0 0) 0x7fea6c02e3e0 con 0x7fea74110bf0 2024-05-27T06:57:40.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:40.418 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:56:40.328992+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:57:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 msgr2=0x7fea580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 0x7fea580b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea64005950 tx=0x7fea6400b410 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 msgr2=0x7fea741a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006070 tx=0x7fea6c004c80 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 shutdown_connections 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fea580b2910 0x7fea580b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea74110bf0 0x7fea741a1770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea74075af0 0x7fea7419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 --2- 172.21.15.16:0/2409041137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea74076440 0x7fea7419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.429+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 >> 172.21.15.16:0/2409041137 conn(0x7fea74100030 msgr2=0x7fea74100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.430+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 shutdown_connections 2024-05-27T06:57:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:40.430+0000 7fea78a00700 1 -- 172.21.15.16:0/2409041137 wait complete. 2024-05-27T06:57:41.332 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T06:57:41.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:57:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:41 smithi164 ceph-mon[35327]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:41 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2409041137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:41.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:41 smithi016 ceph-mon[31415]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:41.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:41 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2409041137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:41 smithi069 ceph-mon[35276]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:41.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:41 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2409041137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:43.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:43 smithi016 ceph-mon[31415]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:43.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:43 smithi164 ceph-mon[35327]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:43.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:43 smithi069 ceph-mon[35276]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 -- 172.21.15.16:0/533264139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488108f10 msgr2=0x7f6488109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 --2- 172.21.15.16:0/533264139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488108f10 0x7f6488109320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64740099c0 tx=0x7f6474009cd0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 -- 172.21.15.16:0/533264139 shutdown_connections 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 --2- 172.21.15.16:0/533264139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488109860 0x7f64881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 --2- 172.21.15.16:0/533264139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 0x7f64881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 --2- 172.21.15.16:0/533264139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488108f10 0x7f6488109320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 -- 172.21.15.16:0/533264139 >> 172.21.15.16:0/533264139 conn(0x7f6488100030 msgr2=0x7f6488102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 -- 172.21.15.16:0/533264139 shutdown_connections 2024-05-27T06:57:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.190+0000 7f648ca00700 1 -- 172.21.15.16:0/533264139 wait complete. 2024-05-27T06:57:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.191+0000 7f648ca00700 1 Processor -- start 2024-05-27T06:57:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 -- start start 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 0x7f648819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488108f10 0x7f648819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6488114370 con 0x7f6488109860 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64881141f0 con 0x7f6488108f10 2024-05-27T06:57:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f648ca00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64881144f0 con 0x7f64881045e0 2024-05-27T06:57:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.192+0000 7f6486800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59800/0 (socket says 172.21.15.16:59800) 2024-05-27T06:57:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 -- 172.21.15.16:0/2320301404 learned_addr learned my addr 172.21.15.16:0/2320301404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6485e00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 0x7f648819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6485400700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488108f10 0x7f648819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 -- 172.21.15.16:0/2320301404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 msgr2=0x7f648819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 0x7f648819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 -- 172.21.15.16:0/2320301404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488108f10 msgr2=0x7f648819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488108f10 0x7f648819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 -- 172.21.15.16:0/2320301404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64881a1e50 con 0x7f6488109860 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.193+0000 7f6486800700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6480007ee0 tx=0x7f648000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.194+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648000dbd0 con 0x7f6488109860 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.194+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f648000dd70 con 0x7f6488109860 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.194+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64800058d0 con 0x7f6488109860 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.194+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64881a2140 con 0x7f6488109860 2024-05-27T06:57:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.194+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64881a9a60 con 0x7f6488109860 2024-05-27T06:57:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.195+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64880512b0 con 0x7f6488109860 2024-05-27T06:57:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.198+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f648001a020 con 0x7f6488109860 2024-05-27T06:57:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.199+0000 7f647ea00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 0x7f646c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.199+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f64800e2580 con 0x7f6488109860 2024-05-27T06:57:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.200+0000 7f6485e00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 0x7f646c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.201+0000 7f6485e00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 0x7f646c0b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6478005fd0 tx=0x7f647800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.202+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6480010070 con 0x7f6488109860 2024-05-27T06:57:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.445+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64881a9c30 con 0x7f6488109860 2024-05-27T06:57:44.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.448+0000 7f647ea00700 1 -- 172.21.15.16:0/2320301404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24708 (secure 0 0 0) 0x7f64800a9d30 con 0x7f6488109860 2024-05-27T06:57:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:57:44.441 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:56:40.328992+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:57:44.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 msgr2=0x7f646c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:44.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 0x7f646c0b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6478005fd0 tx=0x7f647800a560 comp rx=0 tx=0).stop 2024-05-27T06:57:44.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 msgr2=0x7f64881a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:44.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6480007ee0 tx=0x7f648000d3b0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 shutdown_connections 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f646c0b2990 0x7f646c0b4ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6488109860 0x7f64881a17b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64881045e0 0x7f648819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 --2- 172.21.15.16:0/2320301404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6488108f10 0x7f648819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.451+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 >> 172.21.15.16:0/2320301404 conn(0x7f6488100030 msgr2=0x7f648810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.452+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 shutdown_connections 2024-05-27T06:57:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:44.452+0000 7f648ca00700 1 -- 172.21.15.16:0/2320301404 wait complete. 2024-05-27T06:57:45.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T06:57:45.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T06:57:45.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T06:57:45.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T06:57:45.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T06:57:45.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T06:57:45.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-27T06:57:45.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-27T06:57:45.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-27T06:57:45.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-27T06:57:45.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-27T06:57:45.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-27T06:57:45.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:45 smithi016 ceph-mon[31415]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:45.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:45 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2320301404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:45 smithi164 ceph-mon[35327]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:45.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:45 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2320301404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:45.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:45 smithi069 ceph-mon[35276]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:45 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2320301404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:57:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:47.301 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:47.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:47 smithi164 ceph-mon[35327]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:47.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:47 smithi016 ceph-mon[31415]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:47.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:47 smithi069 ceph-mon[35276]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:57:49.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:49 smithi016 ceph-mon[31415]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:49 smithi164 ceph-mon[35327]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:49.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:49 smithi069 ceph-mon[35276]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:51.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:51 smithi016 ceph-mon[31415]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:51.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:51 smithi069 ceph-mon[35276]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:51.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:51 smithi164 ceph-mon[35327]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:57:51.913 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:51.917 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:57:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 -- 172.21.15.16:0/2892746057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b48110960 msgr2=0x7f3b48110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 --2- 172.21.15.16:0/2892746057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b48110960 0x7f3b48110d70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009990 tx=0x7f3b38009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 -- 172.21.15.16:0/2892746057 shutdown_connections 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 --2- 172.21.15.16:0/2892746057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b4810b2d0 0x7f3b48117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 --2- 172.21.15.16:0/2892746057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b48110960 0x7f3b48110d70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 --2- 172.21.15.16:0/2892746057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 0x7f3b4810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 -- 172.21.15.16:0/2892746057 >> 172.21.15.16:0/2892746057 conn(0x7f3b48077c70 msgr2=0x7f3b480780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 -- 172.21.15.16:0/2892746057 shutdown_connections 2024-05-27T06:57:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.591+0000 7f3b4da00700 1 -- 172.21.15.16:0/2892746057 wait complete. 2024-05-27T06:57:52.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.593+0000 7f3b4da00700 1 Processor -- start 2024-05-27T06:57:52.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.593+0000 7f3b4da00700 1 -- start start 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 0x7f3b481a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 0x7f3b481a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b481a5cc0 0x7f3b481a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4811a960 con 0x7f3b4810b2d0 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4811a7e0 con 0x7f3b4810a980 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b4da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4811aae0 con 0x7f3b481a5cc0 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b46000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 0x7f3b481a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b46000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 0x7f3b481a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44324/0 (socket says 172.21.15.16:44324) 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b46000700 1 -- 172.21.15.16:0/4067903168 learned_addr learned my addr 172.21.15.16:0/4067903168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.594+0000 7f3b46a00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 0x7f3b481a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b47400700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b481a5cc0 0x7f3b481a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 -- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b481a5cc0 msgr2=0x7f3b481a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b481a5cc0 0x7f3b481a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 -- 172.21.15.16:0/4067903168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 msgr2=0x7f3b481a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 0x7f3b481a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481a6810 con 0x7f3b4810b2d0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46a00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b4810a980 0x7f3b481a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b46000700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 0x7f3b481a1af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400d980 tx=0x7f3b3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.595+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34017070 con 0x7f3b4810b2d0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.596+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b34010460 con 0x7f3b4810b2d0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.596+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400de60 con 0x7f3b4810b2d0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.596+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481a6b00 con 0x7f3b4810b2d0 2024-05-27T06:57:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.596+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481ae250 con 0x7f3b4810b2d0 2024-05-27T06:57:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.597+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b48063f40 con 0x7f3b4810b2d0 2024-05-27T06:57:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.600+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b34010610 con 0x7f3b4810b2d0 2024-05-27T06:57:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b3f400700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3b2c0b2910 0x7f3b2c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3b340e26d0 con 0x7f3b4810b2d0 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b46a00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3b2c0b2910 0x7f3b2c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b3f400700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] conn(0x7f3b2c0bdc40 0x7f3b2c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b2c0c0800 con 0x7f3b2c0bdc40 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.602+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3b340e2a80 con 0x7f3b4810b2d0 2024-05-27T06:57:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.603+0000 7f3b46a00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3b2c0b2910 0x7f3b2c0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098e0 tx=0x7f3b3800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.603+0000 7f3b47400700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] conn(0x7f3b2c0bdc40 0x7f3b2c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.604+0000 7f3b47400700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] conn(0x7f3b2c0bdc40 0x7f3b2c0c0130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.607+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== osd.2 v2:172.21.15.164:6800/3071758957 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b2c0c0800 con 0x7f3b2c0bdc40 2024-05-27T06:57:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.628+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 --> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b480643c0 con 0x7f3b2c0bdc40 2024-05-27T06:57:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.630+0000 7f3b3f400700 1 -- 172.21.15.16:0/4067903168 <== osd.2 v2:172.21.15.164:6800/3071758957 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3b480643c0 con 0x7f3b2c0bdc40 2024-05-27T06:57:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.630+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] conn(0x7f3b2c0bdc40 msgr2=0x7f3b2c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.630+0000 7f3b4da00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] conn(0x7f3b2c0bdc40 0x7f3b2c0c0130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.630+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3b2c0b2910 msgr2=0x7f3b2c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.630+0000 7f3b4da00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3b2c0b2910 0x7f3b2c0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098e0 tx=0x7f3b3800b540 comp rx=0 tx=0).stop 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 msgr2=0x7f3b481a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 --2- 172.21.15.16:0/4067903168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b4810b2d0 0x7f3b481a1af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400d980 tx=0x7f3b3400dd40 comp rx=0 tx=0).stop 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b47400700 1 -- 172.21.15.16:0/4067903168 reap_dead start 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 shutdown_connections 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 >> 172.21.15.16:0/4067903168 conn(0x7f3b48077c70 msgr2=0x7f3b4807bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 shutdown_connections 2024-05-27T06:57:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:52.631+0000 7f3b4da00700 1 -- 172.21.15.16:0/4067903168 wait complete. 2024-05-27T06:57:53.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:53 smithi016 ceph-mon[31415]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:53 smithi069 ceph-mon[35276]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:53.635 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:53 smithi164 ceph-mon[35327]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:57:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 -- 172.21.15.16:0/2339879372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810a7b0 msgr2=0x7f0e3810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 --2- 172.21.15.16:0/2339879372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810a7b0 0x7f0e3810abc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009990 tx=0x7f0e24009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 -- 172.21.15.16:0/2339879372 shutdown_connections 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 --2- 172.21.15.16:0/2339879372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810b100 0x7f0e38117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 --2- 172.21.15.16:0/2339879372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810f6b0 0x7f0e3810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 --2- 172.21.15.16:0/2339879372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810a7b0 0x7f0e3810abc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.802+0000 7f0e37200700 1 -- 172.21.15.16:0/2339879372 >> 172.21.15.16:0/2339879372 conn(0x7f0e38076af0 msgr2=0x7f0e38076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.803+0000 7f0e37200700 1 -- 172.21.15.16:0/2339879372 shutdown_connections 2024-05-27T06:57:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.803+0000 7f0e37200700 1 -- 172.21.15.16:0/2339879372 wait complete. 2024-05-27T06:57:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.804+0000 7f0e37200700 1 Processor -- start 2024-05-27T06:57:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.804+0000 7f0e37200700 1 -- start start 2024-05-27T06:57:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810a7b0 0x7f0e381a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 0x7f0e381a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810f6b0 0x7f0e381a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3811a7a0 con 0x7f0e3810b100 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3811a620 con 0x7f0e3810a7b0 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e37200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3811a920 con 0x7f0e3810f6b0 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.805+0000 7f0e35400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 0x7f0e381a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 0x7f0e381a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44342/0 (socket says 172.21.15.16:44342) 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 -- 172.21.15.16:0/3244323902 learned_addr learned my addr 172.21.15.16:0/3244323902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35e00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810a7b0 0x7f0e381a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e36800700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810f6b0 0x7f0e381a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 -- 172.21.15.16:0/3244323902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810f6b0 msgr2=0x7f0e381a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810f6b0 0x7f0e381a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 -- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810a7b0 msgr2=0x7f0e381a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810a7b0 0x7f0e381a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e381a5fd0 con 0x7f0e3810b100 2024-05-27T06:57:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e36800700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810f6b0 0x7f0e381a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:57:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35e00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e3810a7b0 0x7f0e381a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.806+0000 7f0e35400700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 0x7f0e381a1750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0e24000c00 tx=0x7f0e240036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.807+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24027070 con 0x7f0e3810b100 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.807+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e2400fc80 con 0x7f0e3810b100 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.807+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24022d60 con 0x7f0e3810b100 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.807+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e381a6260 con 0x7f0e3810b100 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.807+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e381a66f0 con 0x7f0e3810b100 2024-05-27T06:57:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.808+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e38063f00 con 0x7f0e3810b100 2024-05-27T06:57:53.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.812+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e2402a070 con 0x7f0e3810b100 2024-05-27T06:57:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.813+0000 7f0e2ea00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0e1c0b2910 0x7f0e1c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.813+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0e240e3b30 con 0x7f0e3810b100 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.814+0000 7f0e35e00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0e1c0b2910 0x7f0e1c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.814+0000 7f0e2ea00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] conn(0x7f0e1c0bdc40 0x7f0e1c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.814+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e2400b0b0 con 0x7f0e1c0bdc40 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.814+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0e240e3ee0 con 0x7f0e3810b100 2024-05-27T06:57:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.815+0000 7f0e35e00700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0e1c0b2910 0x7f0e1c0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005fd0 tx=0x7f0e2800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.821+0000 7f0e36800700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] conn(0x7f0e1c0bdc40 0x7f0e1c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.822+0000 7f0e36800700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] conn(0x7f0e1c0bdc40 0x7f0e1c0c0130 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.824+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== osd.4 v2:172.21.15.69:6808/823147772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e2400b0b0 con 0x7f0e1c0bdc40 2024-05-27T06:57:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.845+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 --> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e38064380 con 0x7f0e1c0bdc40 2024-05-27T06:57:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.846+0000 7f0e2ea00700 1 -- 172.21.15.16:0/3244323902 <== osd.4 v2:172.21.15.69:6808/823147772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e38064380 con 0x7f0e1c0bdc40 2024-05-27T06:57:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] conn(0x7f0e1c0bdc40 msgr2=0x7f0e1c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] conn(0x7f0e1c0bdc40 0x7f0e1c0c0130 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0e1c0b2910 msgr2=0x7f0e1c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f0e1c0b2910 0x7f0e1c0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e28005fd0 tx=0x7f0e2800a5e0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 msgr2=0x7f0e381a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 --2- 172.21.15.16:0/3244323902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3810b100 0x7f0e381a1750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0e24000c00 tx=0x7f0e240036f0 comp rx=0 tx=0).stop 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e36800700 1 -- 172.21.15.16:0/3244323902 reap_dead start 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 shutdown_connections 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 >> 172.21.15.16:0/3244323902 conn(0x7f0e38076af0 msgr2=0x7f0e38108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 shutdown_connections 2024-05-27T06:57:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:53.847+0000 7f0e37200700 1 -- 172.21.15.16:0/3244323902 wait complete. 2024-05-27T06:57:55.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 -- 172.21.15.16:0/1220823888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf8106560 msgr2=0x7f3cf8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1220823888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf8106560 0x7f3cf8106970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009990 tx=0x7f3ce4009ca0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 -- 172.21.15.16:0/1220823888 shutdown_connections 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1220823888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cf8106eb0 0x7f3cf8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1220823888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf810c5a0 0x7f3cf810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1220823888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf8106560 0x7f3cf8106970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 -- 172.21.15.16:0/1220823888 >> 172.21.15.16:0/1220823888 conn(0x7f3cf8102040 msgr2=0x7f3cf8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 -- 172.21.15.16:0/1220823888 shutdown_connections 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.185+0000 7f3cfe400700 1 -- 172.21.15.16:0/1220823888 wait complete. 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.186+0000 7f3cfe400700 1 Processor -- start 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.186+0000 7f3cfe400700 1 -- start start 2024-05-27T06:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8106560 0x7f3cf819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cf8106eb0 0x7f3cf819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 0x7f3cf81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8116580 con 0x7f3cf810c5a0 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8116400 con 0x7f3cf8106eb0 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cfe400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8116700 con 0x7f3cf8106560 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cf7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 0x7f3cf81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cf7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 0x7f3cf81a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44366/0 (socket says 172.21.15.16:44366) 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.187+0000 7f3cf7e00700 1 -- 172.21.15.16:0/1507536162 learned_addr learned my addr 172.21.15.16:0/1507536162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8106560 0x7f3cf819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf6a00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cf8106eb0 0x7f3cf819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 -- 172.21.15.16:0/1507536162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8106560 msgr2=0x7f3cf819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8106560 0x7f3cf819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 -- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cf8106eb0 msgr2=0x7f3cf819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cf8106eb0 0x7f3cf819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf81a1e00 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cf7e00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 0x7f3cf81a1760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007ee0 tx=0x7f3cf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.188+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000dbd0 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.189+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf81a20f0 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.189+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cf000dd70 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.189+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00058d0 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.189+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf81a9a10 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.190+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cf8063f40 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.192+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3cf001a020 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cefe00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3cdc0b2910 0x7f3cdc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3cf00e2690 con 0x7f3cf810c5a0 2024-05-27T06:57:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cf7400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3cdc0b2910 0x7f3cdc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cefe00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] conn(0x7f3cdc0bdc40 0x7f3cdc0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cf0012ec0 con 0x7f3cdc0bdc40 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.194+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3cf00e2ae0 con 0x7f3cf810c5a0 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.195+0000 7f3cf7400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3cdc0b2910 0x7f3cdc0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005950 tx=0x7f3ce80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.206+0000 7f3cf6a00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] conn(0x7f3cdc0bdc40 0x7f3cdc0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.207+0000 7f3cf6a00700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] conn(0x7f3cdc0bdc40 0x7f3cdc0c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.209+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== osd.10 v2:172.21.15.69:6824/1283060257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cf0012ec0 con 0x7f3cdc0bdc40 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.227+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 --> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cf80643c0 con 0x7f3cdc0bdc40 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.228+0000 7f3cefe00700 1 -- 172.21.15.16:0/1507536162 <== osd.10 v2:172.21.15.69:6824/1283060257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3cf80643c0 con 0x7f3cdc0bdc40 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] conn(0x7f3cdc0bdc40 msgr2=0x7f3cdc0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] conn(0x7f3cdc0bdc40 0x7f3cdc0c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3cdc0b2910 msgr2=0x7f3cdc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f3cdc0b2910 0x7f3cdc0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005950 tx=0x7f3ce80058e0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 msgr2=0x7f3cf81a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cfe400700 1 --2- 172.21.15.16:0/1507536162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf810c5a0 0x7f3cf81a1760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007ee0 tx=0x7f3cf000d3b0 comp rx=0 tx=0).stop 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.229+0000 7f3cf7e00700 1 -- 172.21.15.16:0/1507536162 reap_dead start 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.230+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 shutdown_connections 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.230+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 >> 172.21.15.16:0/1507536162 conn(0x7f3cf8102040 msgr2=0x7f3cf81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.230+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 shutdown_connections 2024-05-27T06:57:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:54.230+0000 7f3cfe400700 1 -- 172.21.15.16:0/1507536162 wait complete. 2024-05-27T06:57:55.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:55 smithi016 ceph-mon[31415]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:55.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:55 smithi069 ceph-mon[35276]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:55.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:55 smithi164 ceph-mon[35327]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:57:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 -- 172.21.15.16:0/277832610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04110ad0 msgr2=0x7fef04112ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 --2- 172.21.15.16:0/277832610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04110ad0 0x7fef04112ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b0a0 tx=0x7feefc00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 -- 172.21.15.16:0/277832610 shutdown_connections 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 --2- 172.21.15.16:0/277832610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04110ad0 0x7fef04112ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 --2- 172.21.15.16:0/277832610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef04102f20 0x7fef04103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 --2- 172.21.15.16:0/277832610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef04103870 0x7fef04110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 -- 172.21.15.16:0/277832610 >> 172.21.15.16:0/277832610 conn(0x7fef04078ab0 msgr2=0x7fef04078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 -- 172.21.15.16:0/277832610 shutdown_connections 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.989+0000 7fef03e00700 1 -- 172.21.15.16:0/277832610 wait complete. 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.990+0000 7fef03e00700 1 Processor -- start 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 -- start start 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef04102f20 0x7fef0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 0x7fef0419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef04110ad0 0x7fef041a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef04116350 con 0x7fef04103870 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef041161d0 con 0x7fef04110ad0 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef03e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef041164d0 con 0x7fef04102f20 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.991+0000 7fef02000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 0x7fef0419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 0x7fef0419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44378/0 (socket says 172.21.15.16:44378) 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 -- 172.21.15.16:0/1086840958 learned_addr learned my addr 172.21.15.16:0/1086840958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02a00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef04102f20 0x7fef0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef03400700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef04110ad0 0x7fef041a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 -- 172.21.15.16:0/1086840958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef04102f20 msgr2=0x7fef0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef04102f20 0x7fef0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 -- 172.21.15.16:0/1086840958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef04110ad0 msgr2=0x7fef041a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef04110ad0 0x7fef041a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a1b30 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.992+0000 7fef02000700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 0x7fef0419d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feef000c970 tx=0x7feef000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.993+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0007b00 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.993+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feef0007ca0 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.993+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a1e20 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.993+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0018660 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.993+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a2360 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.994+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef04063f40 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.998+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7feef000f450 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.999+0000 7feefb400700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7feee80b2910 0x7feee80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:56.999+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feef00e29d0 con 0x7fef04103870 2024-05-27T06:57:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7fef02a00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7feee80b2910 0x7feee80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7feefb400700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] conn(0x7feee80bdc40 0x7feee80c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:57:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feef0015e70 con 0x7feee80bdc40 2024-05-27T06:57:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7feef00e2d80 con 0x7fef04103870 2024-05-27T06:57:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7fef03400700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] conn(0x7feee80bdc40 0x7feee80c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:57:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.000+0000 7fef02a00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7feee80b2910 0x7feee80b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feef4005950 tx=0x7feef40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.001+0000 7fef03400700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] conn(0x7feee80bdc40 0x7feee80c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:57:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.002+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== osd.3 v2:172.21.15.16:6810/3551792310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feef0015e70 con 0x7feee80bdc40 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.021+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 --> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef040643c0 con 0x7feee80bdc40 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.021+0000 7feefb400700 1 -- 172.21.15.16:0/1086840958 <== osd.3 v2:172.21.15.16:6810/3551792310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef040643c0 con 0x7feee80bdc40 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] conn(0x7feee80bdc40 msgr2=0x7feee80c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] conn(0x7feee80bdc40 0x7feee80c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7feee80b2910 msgr2=0x7feee80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7feee80b2910 0x7feee80b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feef4005950 tx=0x7feef40058e0 comp rx=0 tx=0).stop 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 msgr2=0x7fef0419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 --2- 172.21.15.16:0/1086840958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103870 0x7fef0419d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feef000c970 tx=0x7feef000cd30 comp rx=0 tx=0).stop 2024-05-27T06:57:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03400700 1 -- 172.21.15.16:0/1086840958 reap_dead start 2024-05-27T06:57:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 shutdown_connections 2024-05-27T06:57:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 >> 172.21.15.16:0/1086840958 conn(0x7fef04078ab0 msgr2=0x7fef04111ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:57:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 shutdown_connections 2024-05-27T06:57:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:57:57.022+0000 7fef03e00700 1 -- 172.21.15.16:0/1086840958 wait complete. 2024-05-27T06:57:57.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:57 smithi016 ceph-mon[31415]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:57:57.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:57 smithi069 ceph-mon[35276]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:57:57.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:57 smithi164 ceph-mon[35327]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:57:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:57:59 smithi016 ceph-mon[31415]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:57:59 smithi069 ceph-mon[35276]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:57:59.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:57:59 smithi164 ceph-mon[35327]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:01.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:01 smithi016 ceph-mon[31415]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:01.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:01 smithi069 ceph-mon[35276]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout:77309411450 2024-05-27T06:58:01.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T06:58:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:01 smithi164 ceph-mon[35327]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:01.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 -- 172.21.15.16:0/3678855448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 msgr2=0x7f9c10113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 --2- 172.21.15.16:0/3678855448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10113030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b0a0 tx=0x7f9c0800b3b0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 -- 172.21.15.16:0/3678855448 shutdown_connections 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 --2- 172.21.15.16:0/3678855448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10113030 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 --2- 172.21.15.16:0/3678855448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10075ad0 0x7f9c10075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 --2- 172.21.15.16:0/3678855448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c10076420 0x7f9c101106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 -- 172.21.15.16:0/3678855448 >> 172.21.15.16:0/3678855448 conn(0x7f9c10100030 msgr2=0x7f9c10102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 -- 172.21.15.16:0/3678855448 shutdown_connections 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.970+0000 7f9c16800700 1 -- 172.21.15.16:0/3678855448 wait complete. 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.972+0000 7f9c16800700 1 Processor -- start 2024-05-27T06:58:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.972+0000 7f9c16800700 1 -- start start 2024-05-27T06:58:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.972+0000 7f9c16800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c10075ad0 0x7f9c10104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.972+0000 7f9c16800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10076420 0x7f9c10104870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c16800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10104db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c16800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10116570 con 0x7f9c10110c10 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c16800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101163f0 con 0x7f9c10075ad0 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c16800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101166f0 con 0x7f9c10076420 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43344/0 (socket says 172.21.15.16:43344) 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c0fe00700 1 -- 172.21.15.16:0/1778497301 learned_addr learned my addr 172.21.15.16:0/1778497301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.973+0000 7f9c0f400700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c10075ad0 0x7f9c10104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0fe00700 1 -- 172.21.15.16:0/1778497301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10076420 msgr2=0x7f9c10104870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0ea00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10076420 0x7f9c10104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0fe00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10076420 0x7f9c10104870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0fe00700 1 -- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c10075ad0 msgr2=0x7f9c10104330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0fe00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c10075ad0 0x7f9c10104330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0fe00700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c101ad540 con 0x7f9c10110c10 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.974+0000 7f9c0ea00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c10076420 0x7f9c10104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.976+0000 7f9c0fe00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10104db0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b780 tx=0x7f9c08004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.976+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08021070 con 0x7f9c10110c10 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.976+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c08007b80 con 0x7f9c10110c10 2024-05-27T06:58:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.976+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080052a0 con 0x7f9c10110c10 2024-05-27T06:58:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.976+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101ad770 con 0x7f9c10110c10 2024-05-27T06:58:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.977+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101adc00 con 0x7f9c10110c10 2024-05-27T06:58:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.978+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c10063ec0 con 0x7f9c10110c10 2024-05-27T06:58:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.984+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9c08024070 con 0x7f9c10110c10 2024-05-27T06:58:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.986+0000 7f9c07e00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9bf40b2990 0x7f9bf40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.986+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9c0800f020 con 0x7f9c10110c10 2024-05-27T06:58:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.986+0000 7f9c0f400700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9bf40b2990 0x7f9bf40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.987+0000 7f9c07e00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] conn(0x7f9bf40bdcc0 0x7f9bf40c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.987+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c080075c0 con 0x7f9bf40bdcc0 2024-05-27T06:58:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.987+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9c080e4410 con 0x7f9c10110c10 2024-05-27T06:58:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.987+0000 7f9c0ea00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] conn(0x7f9bf40bdcc0 0x7f9bf40c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.988+0000 7f9c0f400700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9bf40b2990 0x7f9bf40b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009c60 tx=0x7f9c00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.988+0000 7f9c0ea00700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] conn(0x7f9bf40bdcc0 0x7f9bf40c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:01.991+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== osd.7 v2:172.21.15.69:6816/125788904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c080075c0 con 0x7f9bf40bdcc0 2024-05-27T06:58:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.014+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 --> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c10064310 con 0x7f9bf40bdcc0 2024-05-27T06:58:02.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c07e00700 1 -- 172.21.15.16:0/1778497301 <== osd.7 v2:172.21.15.69:6816/125788904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c10064310 con 0x7f9bf40bdcc0 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] conn(0x7f9bf40bdcc0 msgr2=0x7f9bf40c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] conn(0x7f9bf40bdcc0 0x7f9bf40c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9bf40b2990 msgr2=0x7f9bf40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9bf40b2990 0x7f9bf40b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009c60 tx=0x7f9c00009260 comp rx=0 tx=0).stop 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 msgr2=0x7f9c10104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.016+0000 7f9c16800700 1 --2- 172.21.15.16:0/1778497301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10110c10 0x7f9c10104db0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b780 tx=0x7f9c08004d00 comp rx=0 tx=0).stop 2024-05-27T06:58:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.017+0000 7f9c0fe00700 1 -- 172.21.15.16:0/1778497301 reap_dead start 2024-05-27T06:58:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.017+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 shutdown_connections 2024-05-27T06:58:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.017+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 >> 172.21.15.16:0/1778497301 conn(0x7f9c10100030 msgr2=0x7f9c10100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.017+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 shutdown_connections 2024-05-27T06:58:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:02.018+0000 7f9c16800700 1 -- 172.21.15.16:0/1778497301 wait complete. 2024-05-27T06:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout:279172874301 2024-05-27T06:58:02.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-27T06:58:03.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:03 smithi016 ceph-mon[31415]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:03 smithi164 ceph-mon[35327]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:03 smithi069 ceph-mon[35276]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:05.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:05 smithi016 ceph-mon[31415]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:05.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:05 smithi164 ceph-mon[35327]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:05.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:05 smithi069 ceph-mon[35276]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:07.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 -- 172.21.15.16:0/311378338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 msgr2=0x7fb7c8117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 --2- 172.21.15.16:0/311378338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c8117800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b0a0 tx=0x7fb7c000b3b0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 -- 172.21.15.16:0/311378338 shutdown_connections 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 --2- 172.21.15.16:0/311378338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c8117800 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 --2- 172.21.15.16:0/311378338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7c8110960 0x7fb7c8110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 --2- 172.21.15.16:0/311378338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7c810a980 0x7fb7c810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.175+0000 7fb7ce200700 1 -- 172.21.15.16:0/311378338 >> 172.21.15.16:0/311378338 conn(0x7fb7c8077c70 msgr2=0x7fb7c80780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.176+0000 7fb7ce200700 1 -- 172.21.15.16:0/311378338 shutdown_connections 2024-05-27T06:58:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.176+0000 7fb7ce200700 1 -- 172.21.15.16:0/311378338 wait complete. 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.177+0000 7fb7ce200700 1 Processor -- start 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.177+0000 7fb7ce200700 1 -- start start 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7c810a980 0x7fb7c81a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7c8110960 0x7fb7c81a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c811a980 con 0x7fb7c810b2d0 2024-05-27T06:58:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c811a800 con 0x7fb7c8110960 2024-05-27T06:58:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7ce200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c811ab00 con 0x7fb7c810a980 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7c6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7c6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c81a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43394/0 (socket says 172.21.15.16:43394) 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.178+0000 7fb7c6a00700 1 -- 172.21.15.16:0/1928082365 learned_addr learned my addr 172.21.15.16:0/1928082365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c7400700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7c810a980 0x7fb7c81a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c7e00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7c8110960 0x7fb7c81a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 -- 172.21.15.16:0/1928082365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7c810a980 msgr2=0x7fb7c81a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7c810a980 0x7fb7c81a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 -- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7c8110960 msgr2=0x7fb7c81a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7c8110960 0x7fb7c81a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81a6200 con 0x7fb7c810b2d0 2024-05-27T06:58:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.179+0000 7fb7c6a00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c81a18d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400d980 tx=0x7fb7b400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.180+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4017070 con 0x7fb7c810b2d0 2024-05-27T06:58:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.180+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c81a64f0 con 0x7fb7c810b2d0 2024-05-27T06:58:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.180+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7b4010460 con 0x7fb7c810b2d0 2024-05-27T06:58:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.180+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400de60 con 0x7fb7c810b2d0 2024-05-27T06:58:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.180+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81ade10 con 0x7fb7c810b2d0 2024-05-27T06:58:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.181+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7c8063f00 con 0x7fb7c810b2d0 2024-05-27T06:58:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.185+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb7b4010610 con 0x7fb7c810b2d0 2024-05-27T06:58:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.186+0000 7fb7bfe00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb7ac0b2910 0x7fb7ac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.186+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb7b40e2710 con 0x7fb7c810b2d0 2024-05-27T06:58:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7c7400700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb7ac0b2910 0x7fb7ac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7bfe00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] conn(0x7fb7ac0bdc40 0x7fb7ac0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7b400fec0 con 0x7fb7ac0bdc40 2024-05-27T06:58:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb7b40e2ac0 con 0x7fb7c810b2d0 2024-05-27T06:58:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7c7400700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb7ac0b2910 0x7fb7ac0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005950 tx=0x7fb7b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.187+0000 7fb7c7e00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] conn(0x7fb7ac0bdc40 0x7fb7ac0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.189+0000 7fb7c7e00700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] conn(0x7fb7ac0bdc40 0x7fb7ac0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.191+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== osd.1 v2:172.21.15.69:6800/1028348959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7b400fec0 con 0x7fb7ac0bdc40 2024-05-27T06:58:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.210+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 --> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7c8064300 con 0x7fb7ac0bdc40 2024-05-27T06:58:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.211+0000 7fb7bfe00700 1 -- 172.21.15.16:0/1928082365 <== osd.1 v2:172.21.15.69:6800/1028348959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb7c8064300 con 0x7fb7ac0bdc40 2024-05-27T06:58:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.212+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] conn(0x7fb7ac0bdc40 msgr2=0x7fb7ac0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.212+0000 7fb7ce200700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] conn(0x7fb7ac0bdc40 0x7fb7ac0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb7ac0b2910 msgr2=0x7fb7ac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb7ac0b2910 0x7fb7ac0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005950 tx=0x7fb7b80058e0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 msgr2=0x7fb7c81a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 --2- 172.21.15.16:0/1928082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c810b2d0 0x7fb7c81a18d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400d980 tx=0x7fb7b400dd40 comp rx=0 tx=0).stop 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7c7e00700 1 -- 172.21.15.16:0/1928082365 reap_dead start 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 shutdown_connections 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 >> 172.21.15.16:0/1928082365 conn(0x7fb7c8077c70 msgr2=0x7fb7c807b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 shutdown_connections 2024-05-27T06:58:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:07.213+0000 7fb7ce200700 1 -- 172.21.15.16:0/1928082365 wait complete. 2024-05-27T06:58:07.703 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:07 smithi016 ceph-mon[31415]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:07.759 INFO:teuthology.orchestra.run.smithi016.stdout:206158430294 2024-05-27T06:58:07.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-27T06:58:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:07 smithi164 ceph-mon[35327]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:07.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:07 smithi069 ceph-mon[35276]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:07.910 INFO:teuthology.orchestra.run.smithi016.stdout:107374182513 2024-05-27T06:58:07.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.854+0000 7ff1fe400700 1 -- 172.21.15.16:0/1951354832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106570 msgr2=0x7ff1f8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.854+0000 7ff1fe400700 1 --2- 172.21.15.16:0/1951354832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106570 0x7ff1f8106980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009990 tx=0x7ff1e4009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 -- 172.21.15.16:0/1951354832 shutdown_connections 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 --2- 172.21.15.16:0/1951354832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8106ec0 0x7ff1f8113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 --2- 172.21.15.16:0/1951354832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f810c5b0 0x7ff1f810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 --2- 172.21.15.16:0/1951354832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106570 0x7ff1f8106980 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 -- 172.21.15.16:0/1951354832 >> 172.21.15.16:0/1951354832 conn(0x7ff1f8102070 msgr2=0x7ff1f81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 -- 172.21.15.16:0/1951354832 shutdown_connections 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.855+0000 7ff1fe400700 1 -- 172.21.15.16:0/1951354832 wait complete. 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 Processor -- start 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 -- start start 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8106570 0x7ff1f819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 0x7ff1f819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f810c5b0 0x7ff1f81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8116590 con 0x7ff1f8106ec0 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8116410 con 0x7ff1f8106570 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.856+0000 7ff1fe400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8116710 con 0x7ff1f810c5b0 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 0x7ff1f819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 0x7ff1f819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43372/0 (socket says 172.21.15.16:43372) 2024-05-27T06:58:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 -- 172.21.15.16:0/4282386720 learned_addr learned my addr 172.21.15.16:0/4282386720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f7400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8106570 0x7ff1f819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f7e00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f810c5b0 0x7ff1f81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 -- 172.21.15.16:0/4282386720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f810c5b0 msgr2=0x7ff1f81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f810c5b0 0x7ff1f81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 -- 172.21.15.16:0/4282386720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8106570 msgr2=0x7ff1f819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8106570 0x7ff1f819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.857+0000 7ff1f6a00700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a1e40 con 0x7ff1f8106ec0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1f6a00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 0x7ff1f819d5c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4000c00 tx=0x7ff1e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4027070 con 0x7ff1f8106ec0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e400fc70 con 0x7ff1f8106ec0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4022d40 con 0x7ff1f8106ec0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a20d0 con 0x7ff1f8106ec0 2024-05-27T06:58:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.858+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a2560 con 0x7ff1f8106ec0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.859+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1f8063f40 con 0x7ff1f8106ec0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.863+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff1e402a070 con 0x7ff1f8106ec0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1efe00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff1dc0b2990 0x7ff1dc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff1e40e4170 con 0x7ff1f8106ec0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1f7400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff1dc0b2990 0x7ff1dc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1efe00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] conn(0x7ff1dc0bdb90 0x7ff1dc0c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1e400fac0 con 0x7ff1dc0bdb90 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff1e40e4520 con 0x7ff1f8106ec0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1f7e00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] conn(0x7ff1dc0bdb90 0x7ff1dc0c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1f7400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff1dc0b2990 0x7ff1dc0b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005fd0 tx=0x7ff1e800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.865+0000 7ff1f7e00700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] conn(0x7ff1dc0bdb90 0x7ff1dc0c0080 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.866+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== osd.0 v2:172.21.15.16:6802/4171777191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1e400fac0 con 0x7ff1dc0bdb90 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.885+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 --> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1f8064390 con 0x7ff1dc0bdb90 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.886+0000 7ff1efe00700 1 -- 172.21.15.16:0/4282386720 <== osd.0 v2:172.21.15.16:6802/4171777191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1f8064390 con 0x7ff1dc0bdb90 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.886+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] conn(0x7ff1dc0bdb90 msgr2=0x7ff1dc0c0080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.886+0000 7ff1fe400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] conn(0x7ff1dc0bdb90 0x7ff1dc0c0080 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff1dc0b2990 msgr2=0x7ff1dc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff1dc0b2990 0x7ff1dc0b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005fd0 tx=0x7ff1e800af20 comp rx=0 tx=0).stop 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 msgr2=0x7ff1f819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 --2- 172.21.15.16:0/4282386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f8106ec0 0x7ff1f819d5c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4000c00 tx=0x7ff1e4009f90 comp rx=0 tx=0).stop 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1f7e00700 1 -- 172.21.15.16:0/4282386720 reap_dead start 2024-05-27T06:58:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 shutdown_connections 2024-05-27T06:58:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 >> 172.21.15.16:0/4282386720 conn(0x7ff1f8102070 msgr2=0x7ff1f81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 shutdown_connections 2024-05-27T06:58:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:06.887+0000 7ff1fe400700 1 -- 172.21.15.16:0/4282386720 wait complete. 2024-05-27T06:58:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.452+0000 7fb172e00700 1 -- 172.21.15.16:0/3597810443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c10c5b0 msgr2=0x7fb16c10c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.452+0000 7fb172e00700 1 --2- 172.21.15.16:0/3597810443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c10c5b0 0x7fb16c10c9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009990 tx=0x7fb15c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 -- 172.21.15.16:0/3597810443 shutdown_connections 2024-05-27T06:58:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 --2- 172.21.15.16:0/3597810443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb16c106ec0 0x7fb16c113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 --2- 172.21.15.16:0/3597810443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c10c5b0 0x7fb16c10c9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 --2- 172.21.15.16:0/3597810443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb16c106570 0x7fb16c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 -- 172.21.15.16:0/3597810443 >> 172.21.15.16:0/3597810443 conn(0x7fb16c102050 msgr2=0x7fb16c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 -- 172.21.15.16:0/3597810443 shutdown_connections 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.453+0000 7fb172e00700 1 -- 172.21.15.16:0/3597810443 wait complete. 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.454+0000 7fb172e00700 1 Processor -- start 2024-05-27T06:58:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.454+0000 7fb172e00700 1 -- start start 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb16c106570 0x7fb16c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 0x7fb16c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb16c10c5b0 0x7fb16c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c116610 con 0x7fb16c106ec0 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c116490 con 0x7fb16c106570 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb172e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c116790 con 0x7fb16c10c5b0 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb16b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 0x7fb16c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb16b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 0x7fb16c19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43448/0 (socket says 172.21.15.16:43448) 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb16b400700 1 -- 172.21.15.16:0/4203160100 learned_addr learned my addr 172.21.15.16:0/4203160100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb16be00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb16c106570 0x7fb16c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.455+0000 7fb170e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb16c10c5b0 0x7fb16c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 -- 172.21.15.16:0/4203160100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb16c10c5b0 msgr2=0x7fb16c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb16c10c5b0 0x7fb16c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 -- 172.21.15.16:0/4203160100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb16c106570 msgr2=0x7fb16c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb16c106570 0x7fb16c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1a1eb0 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb16b400700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 0x7fb16c19d580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b4d0 tx=0x7fb15800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.456+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158015020 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.457+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb158011cc0 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.457+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c1a21a0 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.457+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158010c10 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.457+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1a9ac0 con 0x7fb16c106ec0 2024-05-27T06:58:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.458+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb16c063f40 con 0x7fb16c106ec0 2024-05-27T06:58:09.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.461+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb158018020 con 0x7fb16c106ec0 2024-05-27T06:58:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.463+0000 7fb168c00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb1500b2910 0x7fb1500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.463+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb1580e2480 con 0x7fb16c106ec0 2024-05-27T06:58:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.463+0000 7fb16be00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb1500b2910 0x7fb1500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.463+0000 7fb168c00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] conn(0x7fb1500bdc40 0x7fb1500c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.464+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb15801be50 con 0x7fb1500bdc40 2024-05-27T06:58:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.464+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb1580e28d0 con 0x7fb16c106ec0 2024-05-27T06:58:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.464+0000 7fb170e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] conn(0x7fb1500bdc40 0x7fb1500c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.464+0000 7fb16be00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb1500b2910 0x7fb1500b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b5c0 tx=0x7fb15c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.465+0000 7fb170e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] conn(0x7fb1500bdc40 0x7fb1500c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.466+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== osd.9 v2:172.21.15.16:6826/1571284516 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb15801be50 con 0x7fb1500bdc40 2024-05-27T06:58:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.484+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 --> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb16c0643c0 con 0x7fb1500bdc40 2024-05-27T06:58:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb168c00700 1 -- 172.21.15.16:0/4203160100 <== osd.9 v2:172.21.15.16:6826/1571284516 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb16c0643c0 con 0x7fb1500bdc40 2024-05-27T06:58:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] conn(0x7fb1500bdc40 msgr2=0x7fb1500c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] conn(0x7fb1500bdc40 0x7fb1500c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb1500b2910 msgr2=0x7fb1500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fb1500b2910 0x7fb1500b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b5c0 tx=0x7fb15c005ab0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 msgr2=0x7fb16c19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 --2- 172.21.15.16:0/4203160100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16c106ec0 0x7fb16c19d580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b4d0 tx=0x7fb15800b7e0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb170e00700 1 -- 172.21.15.16:0/4203160100 reap_dead start 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 shutdown_connections 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.485+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 >> 172.21.15.16:0/4203160100 conn(0x7fb16c102050 msgr2=0x7fb16c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.486+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 shutdown_connections 2024-05-27T06:58:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.486+0000 7fb172e00700 1 -- 172.21.15.16:0/4203160100 wait complete. 2024-05-27T06:58:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:09 smithi016 ceph-mon[31415]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:58:09.821 INFO:teuthology.orchestra.run.smithi016.stdout:55834574979 2024-05-27T06:58:09.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T06:58:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:09 smithi164 ceph-mon[35327]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:58:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:09 smithi069 ceph-mon[35276]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 -- 172.21.15.16:0/4069692398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc108730 msgr2=0x7f35dc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 --2- 172.21.15.16:0/4069692398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc108730 0x7f35dc110090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35d40099c0 tx=0x7f35d4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 -- 172.21.15.16:0/4069692398 shutdown_connections 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 --2- 172.21.15.16:0/4069692398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc108730 0x7f35dc110090 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 --2- 172.21.15.16:0/4069692398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35dc1034b0 0x7f35dc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 --2- 172.21.15.16:0/4069692398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc107de0 0x7f35dc1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 -- 172.21.15.16:0/4069692398 >> 172.21.15.16:0/4069692398 conn(0x7f35dc0fedb0 msgr2=0x7f35dc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 -- 172.21.15.16:0/4069692398 shutdown_connections 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.287+0000 7f35e3a00700 1 -- 172.21.15.16:0/4069692398 wait complete. 2024-05-27T06:58:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.289+0000 7f35e3a00700 1 Processor -- start 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.289+0000 7f35e3a00700 1 -- start start 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.289+0000 7f35e3a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35dc1034b0 0x7f35dc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e3a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 0x7f35dc19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e3a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc108730 0x7f35dc1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e3a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc113210 con 0x7f35dc107de0 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e3a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc113090 con 0x7f35dc1034b0 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e3a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc113390 con 0x7f35dc108730 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35dbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 0x7f35dc19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35dbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 0x7f35dc19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43424/0 (socket says 172.21.15.16:43424) 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35dbe00700 1 -- 172.21.15.16:0/3450060096 learned_addr learned my addr 172.21.15.16:0/3450060096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e1000700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35dc1034b0 0x7f35dc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.290+0000 7f35e1a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc108730 0x7f35dc1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 -- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc108730 msgr2=0x7f35dc1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc108730 0x7f35dc1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 -- 172.21.15.16:0/3450060096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35dc1034b0 msgr2=0x7f35dc19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35dc1034b0 0x7f35dc19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc1a1d80 con 0x7f35dc107de0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35e1a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35dc108730 0x7f35dc1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35dbe00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 0x7f35dc19d450 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35c800d980 tx=0x7f35c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8017070 con 0x7f35dc107de0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.291+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c8009dc0 con 0x7f35dc107de0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.292+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc1a2070 con 0x7f35dc107de0 2024-05-27T06:58:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.292+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c800f3c0 con 0x7f35dc107de0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.292+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a9990 con 0x7f35dc107de0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.293+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35dc063f40 con 0x7f35dc107de0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.296+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f35c8009930 con 0x7f35dc107de0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.297+0000 7f35d9600700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35c40b2990 0x7f35c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.297+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f35c80e25b0 con 0x7f35dc107de0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.298+0000 7f35e1000700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35c40b2990 0x7f35c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.298+0000 7f35d9600700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] conn(0x7f35c40bdcc0 0x7f35c40c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.298+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35c800fec0 con 0x7f35c40bdcc0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.298+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f35c80e2960 con 0x7f35dc107de0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.298+0000 7f35e1000700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35c40b2990 0x7f35c40b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009550 tx=0x7f35d00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.311+0000 7f35e1a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] conn(0x7f35c40bdcc0 0x7f35c40c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.313+0000 7f35e1a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] conn(0x7f35c40bdcc0 0x7f35c40c01b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.316+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== osd.8 v2:172.21.15.164:6816/4237870285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35c800fec0 con 0x7f35c40bdcc0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.339+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 --> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35dc064340 con 0x7f35c40bdcc0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35d9600700 1 -- 172.21.15.16:0/3450060096 <== osd.8 v2:172.21.15.164:6816/4237870285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f35dc064340 con 0x7f35c40bdcc0 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] conn(0x7f35c40bdcc0 msgr2=0x7f35c40c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] conn(0x7f35c40bdcc0 0x7f35c40c01b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35c40b2990 msgr2=0x7f35c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35c40b2990 0x7f35c40b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009550 tx=0x7f35d00091b0 comp rx=0 tx=0).stop 2024-05-27T06:58:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 msgr2=0x7f35dc19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e3a00700 1 --2- 172.21.15.16:0/3450060096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35dc107de0 0x7f35dc19d450 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35c800d980 tx=0x7f35c800dc90 comp rx=0 tx=0).stop 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.340+0000 7f35e1a00700 1 -- 172.21.15.16:0/3450060096 reap_dead start 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.341+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 shutdown_connections 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.341+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 >> 172.21.15.16:0/3450060096 conn(0x7f35dc0fedb0 msgr2=0x7f35dc10ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.341+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 shutdown_connections 2024-05-27T06:58:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:09.341+0000 7f35e3a00700 1 -- 172.21.15.16:0/3450060096 wait complete. 2024-05-27T06:58:10.325 INFO:teuthology.orchestra.run.smithi016.stdout:133143986282 2024-05-27T06:58:10.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 -- 172.21.15.16:0/3807182857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109860 msgr2=0x7f29601111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 --2- 172.21.15.16:0/3807182857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109860 0x7f29601111c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f295800b0d0 tx=0x7f295800b3e0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 -- 172.21.15.16:0/3807182857 shutdown_connections 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 --2- 172.21.15.16:0/3807182857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109860 0x7f29601111c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 --2- 172.21.15.16:0/3807182857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29601045e0 0x7f29601049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 --2- 172.21.15.16:0/3807182857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960108f10 0x7f2960109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 -- 172.21.15.16:0/3807182857 >> 172.21.15.16:0/3807182857 conn(0x7f2960100030 msgr2=0x7f2960102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 -- 172.21.15.16:0/3807182857 shutdown_connections 2024-05-27T06:58:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.050+0000 7f2964e00700 1 -- 172.21.15.16:0/3807182857 wait complete. 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.051+0000 7f2964e00700 1 Processor -- start 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.051+0000 7f2964e00700 1 -- start start 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29601045e0 0x7f296019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 0x7f296019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960109860 0x7f29601a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2960114340 con 0x7f2960108f10 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29601141c0 con 0x7f2960109860 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2964e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29601144c0 con 0x7f29601045e0 2024-05-27T06:58:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2955400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 0x7f296019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2955400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 0x7f296019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43464/0 (socket says 172.21.15.16:43464) 2024-05-27T06:58:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f2955400700 1 -- 172.21.15.16:0/540367559 learned_addr learned my addr 172.21.15.16:0/540367559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.052+0000 7f295de00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29601045e0 0x7f296019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f295e800700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960109860 0x7f29601a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 -- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29601045e0 msgr2=0x7f296019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29601045e0 0x7f296019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 -- 172.21.15.16:0/540367559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960109860 msgr2=0x7f29601a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960109860 0x7f29601a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29601a1dc0 con 0x7f2960108f10 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f295e800700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2960109860 0x7f29601a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.053+0000 7f2955400700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 0x7f296019d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f294800d980 tx=0x7f294800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.054+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948017070 con 0x7f2960108f10 2024-05-27T06:58:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.054+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2948009dc0 con 0x7f2960108f10 2024-05-27T06:58:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.054+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294800f3c0 con 0x7f2960108f10 2024-05-27T06:58:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.054+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29601a20b0 con 0x7f2960108f10 2024-05-27T06:58:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.054+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601a99d0 con 0x7f2960108f10 2024-05-27T06:58:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.055+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29601089d0 con 0x7f2960108f10 2024-05-27T06:58:11.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.058+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2948009930 con 0x7f2960108f10 2024-05-27T06:58:11.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.059+0000 7f2957400700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f29440b2910 0x7f29440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.060+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f29480e2650 con 0x7f2960108f10 2024-05-27T06:58:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.060+0000 7f295de00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f29440b2910 0x7f29440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.060+0000 7f2957400700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] conn(0x7f29440bdc40 0x7f29440c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.060+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f294800fec0 con 0x7f29440bdc40 2024-05-27T06:58:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.060+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f29480e2a00 con 0x7f2960108f10 2024-05-27T06:58:11.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.061+0000 7f295de00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f29440b2910 0x7f29440b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2950005950 tx=0x7f295000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:11.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.061+0000 7f295e800700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] conn(0x7f29440bdc40 0x7f29440c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:11.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.062+0000 7f295e800700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] conn(0x7f29440bdc40 0x7f29440c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.065+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== osd.5 v2:172.21.15.164:6808/3562698825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f294800fec0 con 0x7f29440bdc40 2024-05-27T06:58:11.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.083+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 --> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2960105a90 con 0x7f29440bdc40 2024-05-27T06:58:11.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.084+0000 7f2957400700 1 -- 172.21.15.16:0/540367559 <== osd.5 v2:172.21.15.164:6808/3562698825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2960105a90 con 0x7f29440bdc40 2024-05-27T06:58:11.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] conn(0x7f29440bdc40 msgr2=0x7f29440c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] conn(0x7f29440bdc40 0x7f29440c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f29440b2910 msgr2=0x7f29440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f29440b2910 0x7f29440b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2950005950 tx=0x7f295000b410 comp rx=0 tx=0).stop 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 msgr2=0x7f296019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f2964e00700 1 --2- 172.21.15.16:0/540367559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960108f10 0x7f296019d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f294800d980 tx=0x7f294800dc90 comp rx=0 tx=0).stop 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.085+0000 7f295e800700 1 -- 172.21.15.16:0/540367559 reap_dead start 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.086+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 shutdown_connections 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.086+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 >> 172.21.15.16:0/540367559 conn(0x7f2960100030 msgr2=0x7f296010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:11.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.086+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 shutdown_connections 2024-05-27T06:58:11.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:11.086+0000 7f2964e00700 1 -- 172.21.15.16:0/540367559 wait complete. 2024-05-27T06:58:11.372 INFO:teuthology.orchestra.run.smithi016.stdout:34359738505 2024-05-27T06:58:11.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T06:58:11.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:11 smithi016 ceph-mon[31415]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:58:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:11 smithi164 ceph-mon[35327]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:58:11.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:11 smithi069 ceph-mon[35276]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-27T06:58:12.386 INFO:teuthology.orchestra.run.smithi016.stdout:154618822758 2024-05-27T06:58:12.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T06:58:13.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:13 smithi016 ceph-mon[31415]: pgmap v487: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:13 smithi164 ceph-mon[35327]: pgmap v487: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:13.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:13 smithi069 ceph-mon[35276]: pgmap v487: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:14.168 INFO:teuthology.orchestra.run.smithi016.stdout:231928234062 2024-05-27T06:58:14.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 -- 172.21.15.16:0/234298651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce4106570 msgr2=0x7f7ce4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 --2- 172.21.15.16:0/234298651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce4106570 0x7f7ce4106980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009990 tx=0x7f7cd0009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 -- 172.21.15.16:0/234298651 shutdown_connections 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 --2- 172.21.15.16:0/234298651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ce4106ec0 0x7f7ce4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 --2- 172.21.15.16:0/234298651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ce410c5b0 0x7f7ce410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 --2- 172.21.15.16:0/234298651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce4106570 0x7f7ce4106980 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 -- 172.21.15.16:0/234298651 >> 172.21.15.16:0/234298651 conn(0x7f7ce4102070 msgr2=0x7f7ce41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 -- 172.21.15.16:0/234298651 shutdown_connections 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.020+0000 7f7cea200700 1 -- 172.21.15.16:0/234298651 wait complete. 2024-05-27T06:58:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.021+0000 7f7cea200700 1 Processor -- start 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 -- start start 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ce4106570 0x7f7ce419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ce4106ec0 0x7f7ce419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 0x7f7ce41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4116590 con 0x7f7ce410c5b0 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4116410 con 0x7f7ce4106ec0 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7cea200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4116710 con 0x7f7ce4106570 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.022+0000 7f7ce3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 0x7f7ce41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 0x7f7ce41a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56588/0 (socket says 172.21.15.16:56588) 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 -- 172.21.15.16:0/4033551423 learned_addr learned my addr 172.21.15.16:0/4033551423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3400700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ce4106570 0x7f7ce419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce2a00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ce4106ec0 0x7f7ce419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 -- 172.21.15.16:0/4033551423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ce4106570 msgr2=0x7f7ce419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ce4106570 0x7f7ce419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 -- 172.21.15.16:0/4033551423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ce4106ec0 msgr2=0x7f7ce419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ce4106ec0 0x7f7ce419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce41a1e40 con 0x7f7ce410c5b0 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.023+0000 7f7ce3e00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 0x7f7ce41a1850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00ada0 tx=0x7f7cdc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.024+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc017020 con 0x7f7ce410c5b0 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.024+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc00ce10 con 0x7f7ce410c5b0 2024-05-27T06:58:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.024+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc005440 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.024+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce41a2130 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.024+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce41a2670 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.025+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ce4063f40 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.028+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7cdc004ad0 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.029+0000 7f7cdbe00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7cc80b2990 0x7f7cc80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7cdc0e28b0 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7ce3400700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7cc80b2990 0x7f7cc80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7cdbe00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] conn(0x7f7cc80bdcc0 0x7f7cc80c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cc80c0880 con 0x7f7cc80bdcc0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7cdc0e2d00 con 0x7f7ce410c5b0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.030+0000 7f7ce2a00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] conn(0x7f7cc80bdcc0 0x7f7cc80c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.031+0000 7f7ce3400700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7cc80b2990 0x7f7cc80b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4005fd0 tx=0x7f7cd400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.031+0000 7f7ce2a00700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] conn(0x7f7cc80bdcc0 0x7f7cc80c01b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.033+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== osd.6 v2:172.21.15.16:6818/1434782901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cc80c0880 con 0x7f7cc80bdcc0 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.051+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 --> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ce40643c0 con 0x7f7cc80bdcc0 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.052+0000 7f7cdbe00700 1 -- 172.21.15.16:0/4033551423 <== osd.6 v2:172.21.15.16:6818/1434782901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ce40643c0 con 0x7f7cc80bdcc0 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] conn(0x7f7cc80bdcc0 msgr2=0x7f7cc80c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] conn(0x7f7cc80bdcc0 0x7f7cc80c01b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7cc80b2990 msgr2=0x7f7cc80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7cc80b2990 0x7f7cc80b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4005fd0 tx=0x7f7cd400af20 comp rx=0 tx=0).stop 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 msgr2=0x7f7ce41a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7cea200700 1 --2- 172.21.15.16:0/4033551423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce410c5b0 0x7f7ce41a1850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00ada0 tx=0x7f7cdc00c330 comp rx=0 tx=0).stop 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.053+0000 7f7ce3e00700 1 -- 172.21.15.16:0/4033551423 reap_dead start 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.054+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 shutdown_connections 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.054+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 >> 172.21.15.16:0/4033551423 conn(0x7f7ce4102070 msgr2=0x7f7ce41049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.054+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 shutdown_connections 2024-05-27T06:58:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.054+0000 7f7cea200700 1 -- 172.21.15.16:0/4033551423 wait complete. 2024-05-27T06:58:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 -- 172.21.15.16:0/2382124155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 msgr2=0x7fc104114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 --2- 172.21.15.16:0/2382124155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc104114c30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009990 tx=0x7fc0ec009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 -- 172.21.15.16:0/2382124155 shutdown_connections 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 --2- 172.21.15.16:0/2382124155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc104115170 0x7fc104117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 --2- 172.21.15.16:0/2382124155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc104075b50 0x7fc104075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 --2- 172.21.15.16:0/2382124155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc104114c30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 -- 172.21.15.16:0/2382124155 >> 172.21.15.16:0/2382124155 conn(0x7fc104079d00 msgr2=0x7fc10407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 -- 172.21.15.16:0/2382124155 shutdown_connections 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.957+0000 7fc103e00700 1 -- 172.21.15.16:0/2382124155 wait complete. 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.958+0000 7fc103e00700 1 Processor -- start 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.958+0000 7fc103e00700 1 -- start start 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc104075b50 0x7fc1041a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc1041a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc104115170 0x7fc1041a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc10411a730 con 0x7fc1040764a0 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc10411a5b0 con 0x7fc104115170 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc103e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc10411a8b0 con 0x7fc104075b50 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc0fa000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc1041a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc0fa000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc1041a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56606/0 (socket says 172.21.15.16:56606) 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.959+0000 7fc0fa000700 1 -- 172.21.15.16:0/769033266 learned_addr learned my addr 172.21.15.16:0/769033266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc102a00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc104075b50 0x7fc1041a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc103400700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc104115170 0x7fc1041a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 -- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc104075b50 msgr2=0x7fc1041a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc104075b50 0x7fc1041a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 -- 172.21.15.16:0/769033266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc104115170 msgr2=0x7fc1041a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc104115170 0x7fc1041a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1041a6050 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.960+0000 7fc0fa000700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc1041a1720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000c00 tx=0x7fc0ec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.961+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec027070 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.961+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0ec00fb80 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.961+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec022ca0 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.961+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1041a62e0 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.961+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1041adc60 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.962+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc104063f40 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.965+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc0ec02a070 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc0fbe00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc0e80b2990 0x7fc0e80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc0ec0e3a90 con 0x7fc1040764a0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc102a00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc0e80b2990 0x7fc0e80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc0fbe00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] conn(0x7fc0e80bdcc0 0x7fc0e80c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0ec012ec0 con 0x7fc0e80bdcc0 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.966+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc0ec0e3e40 con 0x7fc1040764a0 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.967+0000 7fc103400700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] conn(0x7fc0e80bdcc0 0x7fc0e80c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.967+0000 7fc102a00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc0e80b2990 0x7fc0e80b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005950 tx=0x7fc0f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.968+0000 7fc103400700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] conn(0x7fc0e80bdcc0 0x7fc0e80c01b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.971+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== osd.11 v2:172.21.15.164:6824/3936768359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0ec012ec0 con 0x7fc0e80bdcc0 2024-05-27T06:58:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.990+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 --> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc104064370 con 0x7fc0e80bdcc0 2024-05-27T06:58:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.991+0000 7fc0fbe00700 1 -- 172.21.15.16:0/769033266 <== osd.11 v2:172.21.15.164:6824/3936768359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc104064370 con 0x7fc0e80bdcc0 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.991+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] conn(0x7fc0e80bdcc0 msgr2=0x7fc0e80c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.991+0000 7fc103e00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] conn(0x7fc0e80bdcc0 0x7fc0e80c01b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.991+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc0e80b2990 msgr2=0x7fc0e80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.991+0000 7fc103e00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc0e80b2990 0x7fc0e80b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005950 tx=0x7fc0f40058e0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 msgr2=0x7fc1041a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 --2- 172.21.15.16:0/769033266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1040764a0 0x7fc1041a1720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000c00 tx=0x7fc0ec005ef0 comp rx=0 tx=0).stop 2024-05-27T06:58:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103400700 1 -- 172.21.15.16:0/769033266 reap_dead start 2024-05-27T06:58:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 shutdown_connections 2024-05-27T06:58:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 >> 172.21.15.16:0/769033266 conn(0x7fc104079d00 msgr2=0x7fc104100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 shutdown_connections 2024-05-27T06:58:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:14.992+0000 7fc103e00700 1 -- 172.21.15.16:0/769033266 wait complete. 2024-05-27T06:58:15.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:15 smithi016 ceph-mon[31415]: pgmap v488: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-27T06:58:15.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:15 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:58:15.787 INFO:teuthology.orchestra.run.smithi016.stdout:180388626527 2024-05-27T06:58:15.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-27T06:58:15.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:15 smithi164 ceph-mon[35327]: pgmap v488: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-27T06:58:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:15 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:58:15.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:15 smithi069 ceph-mon[35276]: pgmap v488: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-27T06:58:15.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:15 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:58:16.773 INFO:teuthology.orchestra.run.smithi016.stdout:304942678074 2024-05-27T06:58:16.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-27T06:58:16.893 INFO:teuthology.orchestra.run.smithi016.stdout:257698037832 2024-05-27T06:58:16.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-27T06:58:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:17.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:17 smithi016 ceph-mon[31415]: pgmap v489: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T06:58:17.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:58:17.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:58:17.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:58:17.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:17 smithi164 ceph-mon[35327]: pgmap v489: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T06:58:17.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:58:17.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:58:17.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:58:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:17 smithi069 ceph-mon[35276]: pgmap v489: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T06:58:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:58:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:58:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:58:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:19 smithi016 ceph-mon[31415]: pgmap v490: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:19 smithi164 ceph-mon[35327]: pgmap v490: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:19 smithi069 ceph-mon[35276]: pgmap v490: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:19.870 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:21.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:21 smithi016 ceph-mon[31415]: pgmap v491: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:21.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:21 smithi164 ceph-mon[35327]: pgmap v491: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:21.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:21 smithi069 ceph-mon[35276]: pgmap v491: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:23.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:23 smithi164 ceph-mon[35327]: pgmap v492: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-27T06:58:23.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:23 smithi016 ceph-mon[31415]: pgmap v492: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-27T06:58:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:23 smithi069 ceph-mon[35276]: pgmap v492: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 -- 172.21.15.16:0/1389601967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108d20 msgr2=0x7ff0f0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 --2- 172.21.15.16:0/1389601967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108d20 0x7ff0f0109130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0099c0 tx=0x7ff0dc009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 -- 172.21.15.16:0/1389601967 shutdown_connections 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 --2- 172.21.15.16:0/1389601967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0109670 0x7ff0f0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 --2- 172.21.15.16:0/1389601967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 0x7ff0f0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 --2- 172.21.15.16:0/1389601967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108d20 0x7ff0f0109130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 -- 172.21.15.16:0/1389601967 >> 172.21.15.16:0/1389601967 conn(0x7ff0f0079d00 msgr2=0x7ff0f007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 -- 172.21.15.16:0/1389601967 shutdown_connections 2024-05-27T06:58:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.574+0000 7ff0f6800700 1 -- 172.21.15.16:0/1389601967 wait complete. 2024-05-27T06:58:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.575+0000 7ff0f6800700 1 Processor -- start 2024-05-27T06:58:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 -- start start 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 0x7ff0f019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0108d20 0x7ff0f019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0114180 con 0x7ff0f0109670 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0114000 con 0x7ff0f01043f0 2024-05-27T06:58:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.576+0000 7ff0f6800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0114300 con 0x7ff0f0108d20 2024-05-27T06:58:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57052/0 (socket says 172.21.15.16:57052) 2024-05-27T06:58:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 -- 172.21.15.16:0/423997095 learned_addr learned my addr 172.21.15.16:0/423997095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5400700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 0x7ff0f019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f4a00700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0108d20 0x7ff0f019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 -- 172.21.15.16:0/423997095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0108d20 msgr2=0x7ff0f019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0108d20 0x7ff0f019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 -- 172.21.15.16:0/423997095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 msgr2=0x7ff0f019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 0x7ff0f019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.577+0000 7ff0f5e00700 1 -- 172.21.15.16:0/423997095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a1d70 con 0x7ff0f0109670 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0f5e00700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8007ee0 tx=0x7ff0e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800ddd0 con 0x7ff0f0109670 2024-05-27T06:58:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0e8004020 con 0x7ff0f0109670 2024-05-27T06:58:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a2060 con 0x7ff0f0109670 2024-05-27T06:58:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8015660 con 0x7ff0f0109670 2024-05-27T06:58:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.578+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a9980 con 0x7ff0f0109670 2024-05-27T06:58:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.580+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f01058a0 con 0x7ff0f0109670 2024-05-27T06:58:24.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.582+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff0e80041d0 con 0x7ff0f0109670 2024-05-27T06:58:24.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.584+0000 7ff0ee000700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:24.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.584+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff0e80e2910 con 0x7ff0f0109670 2024-05-27T06:58:24.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.584+0000 7ff0f5400700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:24.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.585+0000 7ff0f5400700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005950 tx=0x7ff0e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:24.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.586+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0e8010070 con 0x7ff0f0109670 2024-05-27T06:58:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.832+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff0f01a9b50 con 0x7ff0f0109670 2024-05-27T06:58:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.833+0000 7ff0ee000700 1 -- 172.21.15.16:0/423997095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff0e80aa0c0 con 0x7ff0f0109670 2024-05-27T06:58:24.825 INFO:teuthology.orchestra.run.smithi016.stdout:279172874307 2024-05-27T06:58:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.835+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 msgr2=0x7ff0d80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005950 tx=0x7ff0e400b410 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 msgr2=0x7ff0f01a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8007ee0 tx=0x7ff0e800d3b0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 shutdown_connections 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0109670 0x7ff0f01a16d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0f01043f0 0x7ff0f019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 --2- 172.21.15.16:0/423997095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0108d20 0x7ff0f019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 >> 172.21.15.16:0/423997095 conn(0x7ff0f0079d00 msgr2=0x7ff0f010f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 shutdown_connections 2024-05-27T06:58:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:24.836+0000 7ff0f6800700 1 -- 172.21.15.16:0/423997095 wait complete. 2024-05-27T06:58:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:25 smithi016 ceph-mon[31415]: pgmap v493: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:25 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/423997095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T06:58:25.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:25 smithi164 ceph-mon[35327]: pgmap v493: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:25.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:25 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/423997095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T06:58:25.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:25 smithi069 ceph-mon[35276]: pgmap v493: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:25.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:25 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/423997095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T06:58:26.689 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:26 smithi016 ceph-mon[31415]: pgmap v494: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T06:58:26.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:26 smithi164 ceph-mon[35327]: pgmap v494: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T06:58:26.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:26 smithi069 ceph-mon[35276]: pgmap v494: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T06:58:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 -- 172.21.15.16:0/329620358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90076440 msgr2=0x7fab901106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 --2- 172.21.15.16:0/329620358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90076440 0x7fab901106b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009990 tx=0x7fab7c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 -- 172.21.15.16:0/329620358 shutdown_connections 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 --2- 172.21.15.16:0/329620358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90110bf0 0x7fab90113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 --2- 172.21.15.16:0/329620358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90075af0 0x7fab90075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 --2- 172.21.15.16:0/329620358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90076440 0x7fab901106b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 -- 172.21.15.16:0/329620358 >> 172.21.15.16:0/329620358 conn(0x7fab90100030 msgr2=0x7fab90102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 -- 172.21.15.16:0/329620358 shutdown_connections 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.434+0000 7fab97600700 1 -- 172.21.15.16:0/329620358 wait complete. 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.436+0000 7fab97600700 1 Processor -- start 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.436+0000 7fab97600700 1 -- start start 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 0x7fab9019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 0x7fab9019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab901165b0 con 0x7fab90110bf0 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab90116430 con 0x7fab90076440 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab97600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab90116730 con 0x7fab90075af0 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.437+0000 7fab95600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57066/0 (socket says 172.21.15.16:57066) 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 -- 172.21.15.16:0/2689892871 learned_addr learned my addr 172.21.15.16:0/2689892871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab94c00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 0x7fab9019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab8fe00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 0x7fab9019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 -- 172.21.15.16:0/2689892871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 msgr2=0x7fab9019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 0x7fab9019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 -- 172.21.15.16:0/2689892871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 msgr2=0x7fab9019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 0x7fab9019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stdout:77309411457 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 -- 172.21.15.16:0/2689892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab901a1ef0 con 0x7fab90110bf0 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab8fe00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 0x7fab9019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab94c00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 0x7fab9019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.438+0000 7fab95600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab880094f0 tx=0x7fab88007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.439+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88004db0 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.439+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab88005020 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.439+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88015660 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.439+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab901a21e0 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.439+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab901a9b00 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.440+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab900512b0 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.443+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fab880051d0 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.445+0000 7fab8d600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 0x7fab780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.445+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fab880e2210 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.445+0000 7fab94c00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 0x7fab780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.446+0000 7fab94c00700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 0x7fab780b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab84005950 tx=0x7fab840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.450+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab88010070 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.696+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab9002e5e0 con 0x7fab90110bf0 2024-05-27T06:58:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.697+0000 7fab8d600700 1 -- 172.21.15.16:0/2689892871 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab880a99c0 con 0x7fab90110bf0 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.700+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 msgr2=0x7fab780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.700+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 0x7fab780b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab84005950 tx=0x7fab840058e0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.700+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 msgr2=0x7fab901a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.700+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab880094f0 tx=0x7fab88007f90 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 shutdown_connections 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fab780b2910 0x7fab780b4e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90110bf0 0x7fab901a1850 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab90075af0 0x7fab9019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 --2- 172.21.15.16:0/2689892871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab90076440 0x7fab9019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 >> 172.21.15.16:0/2689892871 conn(0x7fab90100030 msgr2=0x7fab901007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 shutdown_connections 2024-05-27T06:58:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:26.701+0000 7fab97600700 1 -- 172.21.15.16:0/2689892871 wait complete. 2024-05-27T06:58:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:27 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2689892871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T06:58:27.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:27 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2689892871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T06:58:27.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:27 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2689892871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T06:58:28.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:28 smithi016 ceph-mon[31415]: pgmap v495: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-27T06:58:28.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:28 smithi164 ceph-mon[35327]: pgmap v495: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-27T06:58:28.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:28 smithi069 ceph-mon[35276]: pgmap v495: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-27T06:58:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 -- 172.21.15.16:0/759451759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c108ef0 msgr2=0x7f2b0c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 --2- 172.21.15.16:0/759451759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c109300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2af40099c0 tx=0x7f2af4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 -- 172.21.15.16:0/759451759 shutdown_connections 2024-05-27T06:58:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 --2- 172.21.15.16:0/759451759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c109840 0x7f2b0c1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 --2- 172.21.15.16:0/759451759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 --2- 172.21.15.16:0/759451759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c109300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.227+0000 7f2b12e00700 1 -- 172.21.15.16:0/759451759 >> 172.21.15.16:0/759451759 conn(0x7f2b0c0ffe90 msgr2=0x7f2b0c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.228+0000 7f2b12e00700 1 -- 172.21.15.16:0/759451759 shutdown_connections 2024-05-27T06:58:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.228+0000 7f2b12e00700 1 -- 172.21.15.16:0/759451759 wait complete. 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 Processor -- start 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 -- start start 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c109840 0x7f2b0c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c114350 con 0x7f2b0c1045c0 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1141d0 con 0x7f2b0c109840 2024-05-27T06:58:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.229+0000 7f2b12e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1144d0 con 0x7f2b0c108ef0 2024-05-27T06:58:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.230+0000 7f2b0be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.230+0000 7f2b0be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57102/0 (socket says 172.21.15.16:57102) 2024-05-27T06:58:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.230+0000 7f2b0be00700 1 -- 172.21.15.16:0/633882219 learned_addr learned my addr 172.21.15.16:0/633882219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.230+0000 7f2b03400700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.230+0000 7f2b10e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c109840 0x7f2b0c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 -- 172.21.15.16:0/633882219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c108ef0 msgr2=0x7f2b0c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 -- 172.21.15.16:0/633882219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c109840 msgr2=0x7f2b0c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c109840 0x7f2b0c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 -- 172.21.15.16:0/633882219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c19fc50 con 0x7f2b0c1045c0 2024-05-27T06:58:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b0be00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d950 tx=0x7f2afc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc016070 con 0x7f2b0c1045c0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2afc00fd20 con 0x7f2b0c1045c0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c19ff40 con 0x7f2b0c1045c0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.231+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00e3b0 con 0x7f2b0c1045c0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.232+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a7860 con 0x7f2b0c1045c0 2024-05-27T06:58:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.233+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c0512b0 con 0x7f2b0c1045c0 2024-05-27T06:58:29.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.235+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2afc019070 con 0x7f2b0c1045c0 2024-05-27T06:58:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.237+0000 7f2b09600700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 0x7f2af00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.237+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2afc0e2270 con 0x7f2b0c1045c0 2024-05-27T06:58:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.237+0000 7f2b03400700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 0x7f2af00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.238+0000 7f2b03400700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 0x7f2af00b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009990 tx=0x7f2af4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.242+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2afc0a9a20 con 0x7f2b0c1045c0 2024-05-27T06:58:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.485+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f2b0c02e5e0 con 0x7f2b0c1045c0 2024-05-27T06:58:29.478 INFO:teuthology.orchestra.run.smithi016.stdout:231928234066 2024-05-27T06:58:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.486+0000 7f2b09600700 1 -- 172.21.15.16:0/633882219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2afc0a9320 con 0x7f2b0c1045c0 2024-05-27T06:58:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 msgr2=0x7f2af00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 0x7f2af00b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009990 tx=0x7f2af4005fb0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 msgr2=0x7f2b0c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d950 tx=0x7f2afc00dc60 comp rx=0 tx=0).stop 2024-05-27T06:58:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 shutdown_connections 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2af00b2910 0x7f2af00b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b0c109840 0x7f2b0c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0c1045c0 0x7f2b0c19ad80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 --2- 172.21.15.16:0/633882219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b0c108ef0 0x7f2b0c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.489+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 >> 172.21.15.16:0/633882219 conn(0x7f2b0c0ffe90 msgr2=0x7f2b0c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.490+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 shutdown_connections 2024-05-27T06:58:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:29.490+0000 7f2b12e00700 1 -- 172.21.15.16:0/633882219 wait complete. 2024-05-27T06:58:29.771 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:29 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2509480049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T06:58:29.771 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:29 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/633882219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T06:58:29.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:29 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2509480049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T06:58:29.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:29 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/633882219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T06:58:29.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:29 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2509480049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T06:58:29.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:29 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/633882219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 -- 172.21.15.16:0/1377126279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 msgr2=0x7fc59c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/1377126279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c104800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc5900099c0 tx=0x7fc590009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 -- 172.21.15.16:0/1377126279 shutdown_connections 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/1377126279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 0x7fc59c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/1377126279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c104800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/1377126279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 0x7fc59c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 -- 172.21.15.16:0/1377126279 >> 172.21.15.16:0/1377126279 conn(0x7fc59c079d00 msgr2=0x7fc59c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 -- 172.21.15.16:0/1377126279 shutdown_connections 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.500+0000 7fc5a2e00700 1 -- 172.21.15.16:0/1377126279 wait complete. 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.501+0000 7fc5a2e00700 1 Processor -- start 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 -- start start 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 0x7fc59c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 0x7fc59c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c114150 con 0x7fc59c1043f0 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c113fd0 con 0x7fc59c109670 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.502+0000 7fc5a2e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1142d0 con 0x7fc59c108d20 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57080/0 (socket says 172.21.15.16:57080) 2024-05-27T06:58:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 -- 172.21.15.16:0/2509480049 learned_addr learned my addr 172.21.15.16:0/2509480049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stdout:55834574983 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a2400700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 0x7fc59c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1000700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 0x7fc59c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 -- 172.21.15.16:0/2509480049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 msgr2=0x7fc59c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 0x7fc59c19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 -- 172.21.15.16:0/2509480049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 msgr2=0x7fc59c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 0x7fc59c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.503+0000 7fc5a1a00700 1 -- 172.21.15.16:0/2509480049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1a1cb0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc5a1a00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc590005f50 tx=0x7fc59000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590027070 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc59000fdd0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c1a1f40 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5900173f0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.504+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1a98c0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.506+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c1058a0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.511+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc59002a070 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.513+0000 7fc59a000700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 0x7fc5840b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.514+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc5900e35e0 con 0x7fc59c1043f0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.514+0000 7fc5a1000700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 0x7fc5840b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.515+0000 7fc5a1000700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 0x7fc5840b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc588005950 tx=0x7fc5880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.519+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc59005fdf0 con 0x7fc59c1043f0 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.765+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc59c19e140 con 0x7fc59c1043f0 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.766+0000 7fc59a000700 1 -- 172.21.15.16:0/2509480049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc590013070 con 0x7fc59c1043f0 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 msgr2=0x7fc5840b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 0x7fc5840b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc588005950 tx=0x7fc5880058e0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 msgr2=0x7fc59c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc590005f50 tx=0x7fc59000f460 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 shutdown_connections 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5840b2990 0x7fc5840b4ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc59c109670 0x7fc59c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1043f0 0x7fc59c19cde0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 --2- 172.21.15.16:0/2509480049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc59c108d20 0x7fc59c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 >> 172.21.15.16:0/2509480049 conn(0x7fc59c079d00 msgr2=0x7fc59c102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 shutdown_connections 2024-05-27T06:58:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:28.770+0000 7fc5a2e00700 1 -- 172.21.15.16:0/2509480049 wait complete. 2024-05-27T06:58:30.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:30 smithi016 ceph-mon[31415]: pgmap v496: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:30.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:30 smithi164 ceph-mon[35327]: pgmap v496: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:30.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411450 got 77309411457 for osd.2 2024-05-27T06:58:30.863 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:30.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:30 smithi069 ceph-mon[35276]: pgmap v496: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:58:32.728 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574979 got 55834574983 for osd.1 2024-05-27T06:58:32.728 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:33.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:33 smithi016 ceph-mon[31415]: pgmap v497: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:33.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:33 smithi164 ceph-mon[35327]: pgmap v497: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:33.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:33 smithi069 ceph-mon[35276]: pgmap v497: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:58:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:35 smithi016 ceph-mon[31415]: pgmap v498: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:35 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2350681250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T06:58:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.443+0000 7f36cc800700 1 -- 172.21.15.16:0/1833485882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c4109860 msgr2=0x7f36c41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.443+0000 7f36cc800700 1 --2- 172.21.15.16:0/1833485882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c4109860 0x7f36c41111c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f36c000b0d0 tx=0x7f36c000b3e0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 -- 172.21.15.16:0/1833485882 shutdown_connections 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 --2- 172.21.15.16:0/1833485882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c4109860 0x7f36c41111c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 --2- 172.21.15.16:0/1833485882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41045e0 0x7f36c41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 --2- 172.21.15.16:0/1833485882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 -- 172.21.15.16:0/1833485882 >> 172.21.15.16:0/1833485882 conn(0x7f36c4100030 msgr2=0x7f36c4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 -- 172.21.15.16:0/1833485882 shutdown_connections 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.444+0000 7f36cc800700 1 -- 172.21.15.16:0/1833485882 wait complete. 2024-05-27T06:58:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.445+0000 7f36cc800700 1 Processor -- start 2024-05-27T06:58:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.445+0000 7f36cc800700 1 -- start start 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c4109860 0x7f36c419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4114340 con 0x7f36c41045e0 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41141c0 con 0x7f36c4109860 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36cc800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41144c0 con 0x7f36c4108f10 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36c9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36c9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42204/0 (socket says 172.21.15.16:42204) 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.446+0000 7f36c9e00700 1 -- 172.21.15.16:0/112960761 learned_addr learned my addr 172.21.15.16:0/112960761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36ca800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c4109860 0x7f36c419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9400700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 -- 172.21.15.16:0/112960761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 msgr2=0x7f36c419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 -- 172.21.15.16:0/112960761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c4109860 msgr2=0x7f36c419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c4109860 0x7f36c419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 -- 172.21.15.16:0/112960761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c419fbe0 con 0x7f36c41045e0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9400700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.447+0000 7f36c9e00700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d950 tx=0x7f36b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.448+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8016070 con 0x7f36c41045e0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.448+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b8009dd0 con 0x7f36c41045e0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.448+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c419fed0 con 0x7f36c41045e0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.448+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800f3c0 con 0x7f36c41045e0 2024-05-27T06:58:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.449+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41a77f0 con 0x7f36c41045e0 2024-05-27T06:58:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.451+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c40512b0 con 0x7f36c41045e0 2024-05-27T06:58:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.452+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f36b8019070 con 0x7f36c41045e0 2024-05-27T06:58:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.454+0000 7f36bea00700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 0x7f36ac0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.454+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f36b80e1da0 con 0x7f36c41045e0 2024-05-27T06:58:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.454+0000 7f36c9400700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 0x7f36ac0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.455+0000 7f36c9400700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 0x7f36ac0b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36b4005e10 tx=0x7f36b4005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.462+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36b80a95b0 con 0x7f36c41045e0 2024-05-27T06:58:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:35 smithi164 ceph-mon[35327]: pgmap v498: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:35 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2350681250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T06:58:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:35 smithi069 ceph-mon[35276]: pgmap v498: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:58:35.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:35 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2350681250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T06:58:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.704+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f36c402e5e0 con 0x7f36c41045e0 2024-05-27T06:58:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.705+0000 7f36bea00700 1 -- 172.21.15.16:0/112960761 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f36b80a8eb0 con 0x7f36c41045e0 2024-05-27T06:58:35.698 INFO:teuthology.orchestra.run.smithi016.stdout:107374182521 2024-05-27T06:58:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 msgr2=0x7f36ac0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 0x7f36ac0b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36b4005e10 tx=0x7f36b4005d60 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 msgr2=0x7f36c419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d950 tx=0x7f36b800dc60 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 shutdown_connections 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f36ac0b2990 0x7f36ac0b4ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c4109860 0x7f36c419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36c41045e0 0x7f36c419ad10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 --2- 172.21.15.16:0/112960761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c4108f10 0x7f36c419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 >> 172.21.15.16:0/112960761 conn(0x7f36c4100030 msgr2=0x7f36c4108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 shutdown_connections 2024-05-27T06:58:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.708+0000 7f36cc800700 1 -- 172.21.15.16:0/112960761 wait complete. 2024-05-27T06:58:35.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874301 got 279172874307 for osd.10 2024-05-27T06:58:35.848 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 -- 172.21.15.16:0/2561376390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc10af40 msgr2=0x7fe9fc10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 --2- 172.21.15.16:0/2561376390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc10af40 0x7fe9fc10b350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80099c0 tx=0x7fe9e8009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 -- 172.21.15.16:0/2561376390 shutdown_connections 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 --2- 172.21.15.16:0/2561376390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9fc10b890 0x7fe9fc1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 --2- 172.21.15.16:0/2561376390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc106610 0x7fe9fc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 --2- 172.21.15.16:0/2561376390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc10af40 0x7fe9fc10b350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 -- 172.21.15.16:0/2561376390 >> 172.21.15.16:0/2561376390 conn(0x7fe9fc102020 msgr2=0x7fe9fc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.839+0000 7fea02400700 1 -- 172.21.15.16:0/2561376390 shutdown_connections 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.840+0000 7fea02400700 1 -- 172.21.15.16:0/2561376390 wait complete. 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 Processor -- start 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 -- start start 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 0x7fe9fc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9fc10b890 0x7fe9fc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1163c0 con 0x7fe9fc106610 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc116240 con 0x7fe9fc10af40 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.841+0000 7fea02400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc116540 con 0x7fe9fc10b890 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42188/0 (socket says 172.21.15.16:42188) 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 -- 172.21.15.16:0/2350681250 learned_addr learned my addr 172.21.15.16:0/2350681250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fe9fbe00700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 0x7fe9fc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 -- 172.21.15.16:0/2350681250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9fc10b890 msgr2=0x7fe9fc1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9fc10b890 0x7fe9fc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 -- 172.21.15.16:0/2350681250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 msgr2=0x7fe9fc19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 0x7fe9fc19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 -- 172.21.15.16:0/2350681250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc1a1d00 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fe9fbe00700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 0x7fe9fc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.842+0000 7fea01000700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c920 tx=0x7fe9f000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.843+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0007b00 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.843+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f0004cc0 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.843+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc1a1ff0 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.843+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00054f0 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.843+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1a9910 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.844+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0512b0 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.846+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe9f0007ca0 con 0x7fe9fc106610 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.849+0000 7fe9f9600700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 0x7fe9e40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.849+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe9f00e2760 con 0x7fe9fc106610 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stdout:154618822763 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.849+0000 7fe9fbe00700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 0x7fe9e40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.850+0000 7fe9fbe00700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 0x7fe9e40b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:34.852+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9f0014030 con 0x7fe9fc106610 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.082+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe9fc02e5e0 con 0x7fe9fc106610 2024-05-27T06:58:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.083+0000 7fe9f9600700 1 -- 172.21.15.16:0/2350681250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9f00a9f90 con 0x7fe9fc106610 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.085+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 msgr2=0x7fe9e40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.085+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 0x7fe9e40b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e8005fb0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.085+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 msgr2=0x7fe9fc19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.085+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c920 tx=0x7fe9f000cce0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 shutdown_connections 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9e40b2990 0x7fe9e40b4ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9fc10b890 0x7fe9fc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc106610 0x7fe9fc19ce30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 --2- 172.21.15.16:0/2350681250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9fc10af40 0x7fe9fc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 >> 172.21.15.16:0/2350681250 conn(0x7fe9fc102020 msgr2=0x7fe9fc10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 shutdown_connections 2024-05-27T06:58:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.086+0000 7fea02400700 1 -- 172.21.15.16:0/2350681250 wait complete. 2024-05-27T06:58:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 -- 172.21.15.16:0/1134243238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 msgr2=0x7f3530104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 --2- 172.21.15.16:0/1134243238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f3530104800 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f35240099c0 tx=0x7f3524009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 -- 172.21.15.16:0/1134243238 shutdown_connections 2024-05-27T06:58:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 --2- 172.21.15.16:0/1134243238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f3530110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 --2- 172.21.15.16:0/1134243238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f3530104800 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 --2- 172.21.15.16:0/1134243238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f3530109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 -- 172.21.15.16:0/1134243238 >> 172.21.15.16:0/1134243238 conn(0x7f3530079d00 msgr2=0x7f353007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 -- 172.21.15.16:0/1134243238 shutdown_connections 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.661+0000 7f3536c00700 1 -- 172.21.15.16:0/1134243238 wait complete. 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.662+0000 7f3536c00700 1 Processor -- start 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 -- start start 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f353019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f35301a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35301141a0 con 0x7f35301043f0 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3530114020 con 0x7f3530108d20 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.663+0000 7f3536c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3530114320 con 0x7f3530109670 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.664+0000 7f3535800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f353019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.664+0000 7f3534e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.664+0000 7f3536200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f35301a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.664+0000 7f3534e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:39474/0 (socket says 172.21.15.16:39474) 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.664+0000 7f3534e00700 1 -- 172.21.15.16:0/2066841177 learned_addr learned my addr 172.21.15.16:0/2066841177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 -- 172.21.15.16:0/2066841177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 msgr2=0x7f35301a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f35301a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 -- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 msgr2=0x7f353019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f353019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 -- 172.21.15.16:0/2066841177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35301a1c50 con 0x7f3530108d20 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3536200700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f35301a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.665+0000 7f3534e00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f351c00d980 tx=0x7f351c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.666+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c017070 con 0x7f3530108d20 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.666+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35301a1f40 con 0x7f3530108d20 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.666+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301a9910 con 0x7f3530108d20 2024-05-27T06:58:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.666+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f351c00fd20 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.666+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00e3b0 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.667+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3510005330 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.671+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f351c00f890 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.673+0000 7f352e000700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 0x7f35180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.673+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f351c0e2110 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.674+0000 7f3535800700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 0x7f35180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.674+0000 7f3535800700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 0x7f35180b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3524000c00 tx=0x7f3524005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.686+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f351c0a9940 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.922+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f35100058f0 con 0x7f3530108d20 2024-05-27T06:58:36.458 INFO:teuthology.orchestra.run.smithi016.stdout:133143986290 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.923+0000 7f352e000700 1 -- 172.21.15.16:0/2066841177 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f351c0a9240 con 0x7f3530108d20 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.927+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 msgr2=0x7f35180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.927+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 0x7f35180b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3524000c00 tx=0x7f3524005fb0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.927+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 msgr2=0x7f353019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.927+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f351c00d980 tx=0x7f351c00dc90 comp rx=0 tx=0).stop 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 shutdown_connections 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f35180b2990 0x7f35180b4ea0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530109670 0x7f35301a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35301043f0 0x7f353019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 --2- 172.21.15.16:0/2066841177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530108d20 0x7f353019d3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 >> 172.21.15.16:0/2066841177 conn(0x7f3530079d00 msgr2=0x7f3530101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 shutdown_connections 2024-05-27T06:58:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:35.928+0000 7f3536c00700 1 -- 172.21.15.16:0/2066841177 wait complete. 2024-05-27T06:58:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:36 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/112960761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T06:58:36.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:36 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2066841177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T06:58:36.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:36 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/112960761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T06:58:36.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:36 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2066841177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T06:58:36.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:36 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/112960761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T06:58:36.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:36 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2066841177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T06:58:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:37 smithi164 ceph-mon[35327]: pgmap v499: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:37.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:37 smithi016 ceph-mon[31415]: pgmap v499: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:37.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:37 smithi069 ceph-mon[35276]: pgmap v499: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:58:39.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182513 got 107374182521 for osd.3 2024-05-27T06:58:39.042 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:39.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:39 smithi164 ceph-mon[35327]: pgmap v500: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:39.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:39 smithi016 ceph-mon[31415]: pgmap v500: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:39.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:39 smithi069 ceph-mon[35276]: pgmap v500: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 -- 172.21.15.16:0/3380838417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 msgr2=0x7fa1f41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 --2- 172.21.15.16:0/3380838417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f41111c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0099c0 tx=0x7fa1ec009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 -- 172.21.15.16:0/3380838417 shutdown_connections 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 --2- 172.21.15.16:0/3380838417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f41111c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 --2- 172.21.15.16:0/3380838417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f41045e0 0x7fa1f41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 --2- 172.21.15.16:0/3380838417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f4108f10 0x7fa1f4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 -- 172.21.15.16:0/3380838417 >> 172.21.15.16:0/3380838417 conn(0x7fa1f4100030 msgr2=0x7fa1f4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 -- 172.21.15.16:0/3380838417 shutdown_connections 2024-05-27T06:58:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.795+0000 7fa1fa000700 1 -- 172.21.15.16:0/3380838417 wait complete. 2024-05-27T06:58:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 Processor -- start 2024-05-27T06:58:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 -- start start 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f41045e0 0x7fa1f4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f4108f10 0x7fa1f4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4114390 con 0x7fa1f4109860 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4114210 con 0x7fa1f41045e0 2024-05-27T06:58:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.797+0000 7fa1fa000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4114510 con 0x7fa1f4108f10 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42254/0 (socket says 172.21.15.16:42254) 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 -- 172.21.15.16:0/81057990 learned_addr learned my addr 172.21.15.16:0/81057990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3400700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f41045e0 0x7fa1f4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1eaa00700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f4108f10 0x7fa1f4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 -- 172.21.15.16:0/81057990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f4108f10 msgr2=0x7fa1f4075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f4108f10 0x7fa1f4075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 -- 172.21.15.16:0/81057990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f41045e0 msgr2=0x7fa1f4075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f41045e0 0x7fa1f4075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.798+0000 7fa1f3e00700 1 -- 172.21.15.16:0/81057990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f41a9100 con 0x7fa1f4109860 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1f3e00700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b580 tx=0x7fa1ec00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec027070 con 0x7fa1f4109860 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1ec00fd20 con 0x7fa1f4109860 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec022e30 con 0x7fa1f4109860 2024-05-27T06:58:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f41a9390 con 0x7fa1f4109860 2024-05-27T06:58:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.799+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f41a9840 con 0x7fa1f4109860 2024-05-27T06:58:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.801+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f40512b0 con 0x7fa1f4109860 2024-05-27T06:58:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.804+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa1ec02a070 con 0x7fa1f4109860 2024-05-27T06:58:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.805+0000 7fa1f0c00700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 0x7fa1d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.806+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1ec0e3350 con 0x7fa1f4109860 2024-05-27T06:58:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.806+0000 7fa1f3400700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 0x7fa1d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.807+0000 7fa1f3400700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 0x7fa1d80b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009550 tx=0x7fa1e40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:39.808+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1ec013070 con 0x7fa1f4109860 2024-05-27T06:58:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.073+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa1f402e5e0 con 0x7fa1f4109860 2024-05-27T06:58:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.074+0000 7fa1f0c00700 1 -- 172.21.15.16:0/81057990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa1ec0aadf0 con 0x7fa1f4109860 2024-05-27T06:58:40.069 INFO:teuthology.orchestra.run.smithi016.stdout:206158430301 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 msgr2=0x7fa1d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 0x7fa1d80b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009550 tx=0x7fa1e40091b0 comp rx=0 tx=0).stop 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 msgr2=0x7fa1f4076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b580 tx=0x7fa1ec00f460 comp rx=0 tx=0).stop 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 shutdown_connections 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa1d80b2900 0x7fa1d80b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f4109860 0x7fa1f4076530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f41045e0 0x7fa1f4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 --2- 172.21.15.16:0/81057990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1f4108f10 0x7fa1f4075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 >> 172.21.15.16:0/81057990 conn(0x7fa1f4100030 msgr2=0x7fa1f4108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.077+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 shutdown_connections 2024-05-27T06:58:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:40.078+0000 7fa1fa000700 1 -- 172.21.15.16:0/81057990 wait complete. 2024-05-27T06:58:40.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822758 got 154618822763 for osd.5 2024-05-27T06:58:40.246 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:40.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:40 smithi016 ceph-mon[31415]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-27T06:58:40.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:40 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/81057990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T06:58:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:40 smithi164 ceph-mon[35327]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-27T06:58:40.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:40 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/81057990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T06:58:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:40 smithi069 ceph-mon[35276]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-27T06:58:40.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:40 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/81057990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T06:58:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:41 smithi164 ceph-mon[35327]: pgmap v501: 129 pgs: 129 active+clean; 649 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T06:58:41.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:41 smithi016 ceph-mon[31415]: pgmap v501: 129 pgs: 129 active+clean; 649 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T06:58:41.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:41 smithi069 ceph-mon[35276]: pgmap v501: 129 pgs: 129 active+clean; 649 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T06:58:42.471 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986282 got 133143986290 for osd.4 2024-05-27T06:58:42.472 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.401+0000 7ff84ae00700 1 -- 172.21.15.16:0/1501310496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff844109860 msgr2=0x7ff8441111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.401+0000 7ff84ae00700 1 --2- 172.21.15.16:0/1501310496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff844109860 0x7ff8441111c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b0d0 tx=0x7ff83c00b3e0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 -- 172.21.15.16:0/1501310496 shutdown_connections 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 --2- 172.21.15.16:0/1501310496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff844109860 0x7ff8441111c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 --2- 172.21.15.16:0/1501310496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8441045e0 0x7ff8441049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 --2- 172.21.15.16:0/1501310496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844108f10 0x7ff844109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 -- 172.21.15.16:0/1501310496 >> 172.21.15.16:0/1501310496 conn(0x7ff844100030 msgr2=0x7ff844102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 -- 172.21.15.16:0/1501310496 shutdown_connections 2024-05-27T06:58:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.402+0000 7ff84ae00700 1 -- 172.21.15.16:0/1501310496 wait complete. 2024-05-27T06:58:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.403+0000 7ff84ae00700 1 Processor -- start 2024-05-27T06:58:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 -- start start 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff844108f10 0x7ff844075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 0x7ff844076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff844114340 con 0x7ff8441045e0 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441141c0 con 0x7ff844109860 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff84ae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441144c0 con 0x7ff844108f10 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.404+0000 7ff843e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41760/0 (socket says 172.21.15.16:41760) 2024-05-27T06:58:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 -- 172.21.15.16:0/829527961 learned_addr learned my addr 172.21.15.16:0/829527961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843400700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff844108f10 0x7ff844075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 -- 172.21.15.16:0/829527961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff844108f10 msgr2=0x7ff844075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff844108f10 0x7ff844075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 -- 172.21.15.16:0/829527961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 msgr2=0x7ff844076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff848e00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 0x7ff844076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 0x7ff844076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff843e00700 1 -- 172.21.15.16:0/829527961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8441a9020 con 0x7ff8441045e0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.405+0000 7ff848e00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 0x7ff844076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff843e00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c9b0 tx=0x7ff83400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834007b00 con 0x7ff8441045e0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff834007ca0 con 0x7ff8441045e0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8441a9310 con 0x7ff8441045e0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834018660 con 0x7ff8441045e0 2024-05-27T06:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.406+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8441a98a0 con 0x7ff8441045e0 2024-05-27T06:58:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.407+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8440512b0 con 0x7ff8441045e0 2024-05-27T06:58:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.412+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff83400f450 con 0x7ff8441045e0 2024-05-27T06:58:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.414+0000 7ff840c00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.414+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff8340e2890 con 0x7ff8441045e0 2024-05-27T06:58:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.414+0000 7ff843400700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.415+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff834014030 con 0x7ff8441045e0 2024-05-27T06:58:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.415+0000 7ff843400700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 0x7ff8280b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff830005950 tx=0x7ff8300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 -- 172.21.15.16:0/1994256991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b0105e20 msgr2=0x7fe9b010e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1994256991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b0105e20 0x7fe9b010e370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009990 tx=0x7fe99c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 -- 172.21.15.16:0/1994256991 shutdown_connections 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1994256991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b010e900 0x7fe9b0110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1994256991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b01054d0 0x7fe9b01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1994256991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b0105e20 0x7fe9b010e370 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 -- 172.21.15.16:0/1994256991 >> 172.21.15.16:0/1994256991 conn(0x7fe9b00fef20 msgr2=0x7fe9b0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 -- 172.21.15.16:0/1994256991 shutdown_connections 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.299+0000 7fe9b5400700 1 -- 172.21.15.16:0/1994256991 wait complete. 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.300+0000 7fe9b5400700 1 Processor -- start 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.300+0000 7fe9b5400700 1 -- start start 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b01054d0 0x7fe9b019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b0105e20 0x7fe9b019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01143d0 con 0x7fe9b010e900 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b0114250 con 0x7fe9b01054d0 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9b5400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b0114550 con 0x7fe9b0105e20 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9af200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9af200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41732/0 (socket says 172.21.15.16:41732) 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.301+0000 7fe9af200700 1 -- 172.21.15.16:0/1386797637 learned_addr learned my addr 172.21.15.16:0/1386797637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9ade00700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b0105e20 0x7fe9b019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9ae800700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b01054d0 0x7fe9b019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 -- 172.21.15.16:0/1386797637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b0105e20 msgr2=0x7fe9b019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b0105e20 0x7fe9b019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 -- 172.21.15.16:0/1386797637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b01054d0 msgr2=0x7fe9b019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b01054d0 0x7fe9b019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 -- 172.21.15.16:0/1386797637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b01a1db0 con 0x7fe9b010e900 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.302+0000 7fe9af200700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c900 tx=0x7fe9a800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.303+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8007b00 con 0x7fe9b010e900 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.303+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9a8007ca0 con 0x7fe9b010e900 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.303+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b01a20a0 con 0x7fe9b010e900 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.303+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8018660 con 0x7fe9b010e900 2024-05-27T06:58:43.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.303+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b01a2590 con 0x7fe9b010e900 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.304+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b00512b0 con 0x7fe9b010e900 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.306+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe9a800f450 con 0x7fe9b010e900 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.308+0000 7fe9a7400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 0x7fe9940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.308+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe9a80e2640 con 0x7fe9b010e900 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.308+0000 7fe9ae800700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 0x7fe9940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.309+0000 7fe9ae800700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 0x7fe9940b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f20 tx=0x7fe9a00094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.313+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9a8014030 con 0x7fe9b010e900 2024-05-27T06:58:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.563+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe9b01a9b30 con 0x7fe9b010e900 2024-05-27T06:58:43.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:43 smithi016 ceph-mon[31415]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T06:58:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.565+0000 7fe9a7400700 1 -- 172.21.15.16:0/1386797637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9a80a9f50 con 0x7fe9b010e900 2024-05-27T06:58:43.557 INFO:teuthology.orchestra.run.smithi016.stdout:180388626532 2024-05-27T06:58:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 msgr2=0x7fe9940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 0x7fe9940b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f20 tx=0x7fe9a00094f0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 msgr2=0x7fe9b01a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c900 tx=0x7fe9a800ccc0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 shutdown_connections 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe9940b2880 0x7fe9940b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b010e900 0x7fe9b01a17c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9b01054d0 0x7fe9b019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 --2- 172.21.15.16:0/1386797637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9b0105e20 0x7fe9b019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.569+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 >> 172.21.15.16:0/1386797637 conn(0x7fe9b00fef20 msgr2=0x7fe9b00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.570+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 shutdown_connections 2024-05-27T06:58:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.570+0000 7fe9b5400700 1 -- 172.21.15.16:0/1386797637 wait complete. 2024-05-27T06:58:43.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:43 smithi164 ceph-mon[35327]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T06:58:43.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:43 smithi069 ceph-mon[35276]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T06:58:43.658 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234062 got 231928234066 for osd.8 2024-05-27T06:58:43.658 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.697+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff84402e5e0 con 0x7ff8441045e0 2024-05-27T06:58:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.698+0000 7ff840c00700 1 -- 172.21.15.16:0/829527961 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff8340aa310 con 0x7ff8441045e0 2024-05-27T06:58:43.690 INFO:teuthology.orchestra.run.smithi016.stdout:304942678080 2024-05-27T06:58:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 msgr2=0x7ff8280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 0x7ff8280b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff830005950 tx=0x7ff8300058e0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 msgr2=0x7ff844075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c9b0 tx=0x7ff83400cd70 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 shutdown_connections 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff844109860 0x7ff844076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8441045e0 0x7ff844075ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 --2- 172.21.15.16:0/829527961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff844108f10 0x7ff844075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 >> 172.21.15.16:0/829527961 conn(0x7ff844100030 msgr2=0x7ff844108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 shutdown_connections 2024-05-27T06:58:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:43.701+0000 7ff84ae00700 1 -- 172.21.15.16:0/829527961 wait complete. 2024-05-27T06:58:44.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430294 got 206158430301 for osd.7 2024-05-27T06:58:44.392 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:44.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1386797637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T06:58:44.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/829527961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T06:58:44.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1386797637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T06:58:44.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/829527961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T06:58:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1386797637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T06:58:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/829527961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T06:58:45.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626527 got 180388626532 for osd.6 2024-05-27T06:58:45.259 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:45.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:45 smithi016 ceph-mon[31415]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:45.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:45 smithi069 ceph-mon[35276]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:45 smithi164 ceph-mon[35327]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:46.065 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678074 got 304942678080 for osd.11 2024-05-27T06:58:46.065 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.416+0000 7fe79c200700 1 -- 172.21.15.16:0/358700263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 msgr2=0x7fe794109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.416+0000 7fe79c200700 1 --2- 172.21.15.16:0/358700263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe794109320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe7800099c0 tx=0x7fe780009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 -- 172.21.15.16:0/358700263 shutdown_connections 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 --2- 172.21.15.16:0/358700263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 --2- 172.21.15.16:0/358700263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 0x7fe7941049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 --2- 172.21.15.16:0/358700263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe794109320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 -- 172.21.15.16:0/358700263 >> 172.21.15.16:0/358700263 conn(0x7fe794100030 msgr2=0x7fe794102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 -- 172.21.15.16:0/358700263 shutdown_connections 2024-05-27T06:58:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.417+0000 7fe79c200700 1 -- 172.21.15.16:0/358700263 wait complete. 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.418+0000 7fe79c200700 1 Processor -- start 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.418+0000 7fe79c200700 1 -- start start 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.418+0000 7fe79c200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 0x7fe79419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.418+0000 7fe79c200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.418+0000 7fe79c200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe79c200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe794114370 con 0x7fe794108f10 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe79c200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941141f0 con 0x7fe794109860 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe79c200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941144f0 con 0x7fe7941045e0 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41776/0 (socket says 172.21.15.16:41776) 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 -- 172.21.15.16:0/2758115293 learned_addr learned my addr 172.21.15.16:0/2758115293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe799800700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 0x7fe79419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe79a200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 -- 172.21.15.16:0/2758115293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 msgr2=0x7fe79419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 0x7fe79419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 -- 172.21.15.16:0/2758115293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 msgr2=0x7fe7941a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe798e00700 1 -- 172.21.15.16:0/2758115293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7941a1ed0 con 0x7fe794108f10 2024-05-27T06:58:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.419+0000 7fe79a200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe798e00700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b580 tx=0x7fe78000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780027070 con 0x7fe794108f10 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941a2160 con 0x7fe794108f10 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7800056d0 con 0x7fe794108f10 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800173f0 con 0x7fe794108f10 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.420+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941a2640 con 0x7fe794108f10 2024-05-27T06:58:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.421+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7940512b0 con 0x7fe794108f10 2024-05-27T06:58:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.426+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe78002a070 con 0x7fe794108f10 2024-05-27T06:58:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.427+0000 7fe792000700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 0x7fe77c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.428+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe7800e3820 con 0x7fe794108f10 2024-05-27T06:58:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.428+0000 7fe799800700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 0x7fe77c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.429+0000 7fe799800700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 0x7fe77c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe788005fd0 tx=0x7fe78800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.429+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe78005fdf0 con 0x7fe794108f10 2024-05-27T06:58:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.672+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe7941a9ae0 con 0x7fe794108f10 2024-05-27T06:58:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.673+0000 7fe792000700 1 -- 172.21.15.16:0/2758115293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe780013070 con 0x7fe794108f10 2024-05-27T06:58:46.665 INFO:teuthology.orchestra.run.smithi016.stdout:257698037839 2024-05-27T06:58:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.676+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 msgr2=0x7fe77c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.676+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 0x7fe77c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe788005fd0 tx=0x7fe78800a560 comp rx=0 tx=0).stop 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.676+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 msgr2=0x7fe79419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.676+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b580 tx=0x7fe78000f740 comp rx=0 tx=0).stop 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 shutdown_connections 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe77c0b2880 0x7fe77c0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe794109860 0x7fe7941a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7941045e0 0x7fe79419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 --2- 172.21.15.16:0/2758115293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794108f10 0x7fe79419d5a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 >> 172.21.15.16:0/2758115293 conn(0x7fe794100030 msgr2=0x7fe79410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 shutdown_connections 2024-05-27T06:58:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:46.677+0000 7fe79c200700 1 -- 172.21.15.16:0/2758115293 wait complete. 2024-05-27T06:58:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.054+0000 7f7df6200700 1 -- 172.21.15.16:0/1384814997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df0105e40 msgr2=0x7f7df010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.054+0000 7f7df6200700 1 --2- 172.21.15.16:0/1384814997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df0105e40 0x7f7df010e390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009990 tx=0x7f7dd8009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 -- 172.21.15.16:0/1384814997 shutdown_connections 2024-05-27T06:58:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 --2- 172.21.15.16:0/1384814997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df010e8d0 0x7f7df0110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 --2- 172.21.15.16:0/1384814997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df01054f0 0x7f7df0105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 --2- 172.21.15.16:0/1384814997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df0105e40 0x7f7df010e390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 -- 172.21.15.16:0/1384814997 >> 172.21.15.16:0/1384814997 conn(0x7f7df00fedb0 msgr2=0x7f7df01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 -- 172.21.15.16:0/1384814997 shutdown_connections 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.055+0000 7f7df6200700 1 -- 172.21.15.16:0/1384814997 wait complete. 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.056+0000 7f7df6200700 1 Processor -- start 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 -- start start 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df01054f0 0x7f7df01a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0105e40 0x7f7df01a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01143b0 con 0x7f7df010e8d0 2024-05-27T06:58:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0114230 con 0x7f7df01054f0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7df6200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0114530 con 0x7f7df0105e40 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7defe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7defe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.057+0000 7f7defe00700 1 -- 172.21.15.16:0/3028433976 learned_addr learned my addr 172.21.15.16:0/3028433976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7de7e00700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0105e40 0x7f7df01a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7def400700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df01054f0 0x7f7df01a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 -- 172.21.15.16:0/3028433976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0105e40 msgr2=0x7f7df01a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0105e40 0x7f7df01a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 -- 172.21.15.16:0/3028433976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df01054f0 msgr2=0x7f7df01a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df01054f0 0x7f7df01a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 -- 172.21.15.16:0/3028433976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df01ad650 con 0x7f7df010e8d0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.058+0000 7f7defe00700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b970 tx=0x7f7de800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.059+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8019070 con 0x7f7df010e8d0 2024-05-27T06:58:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.059+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de800ce40 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.059+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01ad820 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.059+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8007370 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.059+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01add10 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.060+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00512b0 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.062+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7de801c070 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.065+0000 7f7decc00700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.065+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7de80e22b0 con 0x7f7df010e8d0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.065+0000 7f7def400700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.066+0000 7f7def400700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 0x7f7dd40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005fd0 tx=0x7f7de000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.067+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7de8023070 con 0x7f7df010e8d0 2024-05-27T06:58:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.294+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7df010df10 con 0x7f7df010e8d0 2024-05-27T06:58:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.295+0000 7f7decc00700 1 -- 172.21.15.16:0/3028433976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7df010df10 con 0x7f7df010e8d0 2024-05-27T06:58:47.288 INFO:teuthology.orchestra.run.smithi016.stdout:34359738513 2024-05-27T06:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.298+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 msgr2=0x7f7dd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.298+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 0x7f7dd40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005fd0 tx=0x7f7de000a560 comp rx=0 tx=0).stop 2024-05-27T06:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.298+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 msgr2=0x7f7df01ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.298+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b970 tx=0x7f7de800bd30 comp rx=0 tx=0).stop 2024-05-27T06:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 shutdown_connections 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df010e8d0 0x7f7df01ad110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df01054f0 0x7f7df01a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 --2- 172.21.15.16:0/3028433976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0105e40 0x7f7df01a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 >> 172.21.15.16:0/3028433976 conn(0x7f7df00fedb0 msgr2=0x7f7df01031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 shutdown_connections 2024-05-27T06:58:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:47.299+0000 7f7df6200700 1 -- 172.21.15.16:0/3028433976 wait complete. 2024-05-27T06:58:47.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:47 smithi016 ceph-mon[31415]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-27T06:58:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:47 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2758115293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T06:58:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:47 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3028433976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T06:58:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:47 smithi069 ceph-mon[35276]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-27T06:58:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:47 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2758115293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T06:58:47.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:47 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3028433976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T06:58:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:47 smithi164 ceph-mon[35327]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-27T06:58:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:47 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2758115293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T06:58:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:47 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3028433976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T06:58:48.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037832 got 257698037839 for osd.9 2024-05-27T06:58:48.654 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:48.851 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738505 got 34359738513 for osd.0 2024-05-27T06:58:48.852 DEBUG:teuthology.parallel:result is None 2024-05-27T06:58:48.852 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T06:58:48.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T06:58:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:49 smithi016 ceph-mon[31415]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:49.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:49 smithi069 ceph-mon[35276]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:49.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:49 smithi164 ceph-mon[35327]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-27T06:58:51.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:51 smithi016 ceph-mon[31415]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-27T06:58:51.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:51 smithi069 ceph-mon[35276]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-27T06:58:51.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:51 smithi164 ceph-mon[35327]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.902+0000 7f347b600700 1 -- 172.21.15.16:0/88657073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 msgr2=0x7f34741049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.902+0000 7f347b600700 1 --2- 172.21.15.16:0/88657073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f34741049d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34680099c0 tx=0x7f3468009cd0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 -- 172.21.15.16:0/88657073 shutdown_connections 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 --2- 172.21.15.16:0/88657073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f34741111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 --2- 172.21.15.16:0/88657073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f34741049d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 --2- 172.21.15.16:0/88657073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 0x7f3474109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 -- 172.21.15.16:0/88657073 >> 172.21.15.16:0/88657073 conn(0x7f34740ffe90 msgr2=0x7f34741022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 -- 172.21.15.16:0/88657073 shutdown_connections 2024-05-27T06:58:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.903+0000 7f347b600700 1 -- 172.21.15.16:0/88657073 wait complete. 2024-05-27T06:58:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.904+0000 7f347b600700 1 Processor -- start 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 -- start start 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 0x7f347419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f347419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3474114320 con 0x7f34741045c0 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34741141a0 con 0x7f3474109840 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f347b600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34741144a0 con 0x7f3474108ef0 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f3478c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f3478c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41500/0 (socket says 172.21.15.16:41500) 2024-05-27T06:58:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.905+0000 7f3478c00700 1 -- 172.21.15.16:0/3549748297 learned_addr learned my addr 172.21.15.16:0/3549748297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3473e00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 0x7f347419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3479600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f347419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 -- 172.21.15.16:0/3549748297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 msgr2=0x7f347419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 0x7f347419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 -- 172.21.15.16:0/3549748297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 msgr2=0x7f347419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f347419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 -- 172.21.15.16:0/3549748297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347419fb30 con 0x7f34741045c0 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3479600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f347419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.906+0000 7f3478c00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3468005f50 tx=0x7f346800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.907+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468027070 con 0x7f34741045c0 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.907+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f346800fdd0 con 0x7f34741045c0 2024-05-27T06:58:51.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.907+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347419fdc0 con 0x7f34741045c0 2024-05-27T06:58:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.907+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680173f0 con 0x7f34741045c0 2024-05-27T06:58:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.907+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34741a02a0 con 0x7f34741045c0 2024-05-27T06:58:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.908+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34740512b0 con 0x7f34741045c0 2024-05-27T06:58:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.911+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f346802a070 con 0x7f34741045c0 2024-05-27T06:58:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.912+0000 7f3471600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 0x7f345c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.913+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f34680e37a0 con 0x7f34741045c0 2024-05-27T06:58:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.913+0000 7f3473e00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 0x7f345c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.914+0000 7f3473e00700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 0x7f345c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3464005ea0 tx=0x7f3464005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:51.915+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f346805fdf0 con 0x7f34741045c0 2024-05-27T06:58:52.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.153+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f347419c070 con 0x7f345c0b2880 2024-05-27T06:58:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.184+0000 7f3471600700 1 -- 172.21.15.16:0/3549748297 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7f347419c070 con 0x7f345c0b2880 2024-05-27T06:58:52.179 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:58:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.193+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 msgr2=0x7f345c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.193+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 0x7f345c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3464005ea0 tx=0x7f3464005e30 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.193+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 msgr2=0x7f347419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.193+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3468005f50 tx=0x7f346800f460 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 shutdown_connections 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f345c0b2880 0x7f345c0b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3474109840 0x7f347419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741045c0 0x7f347419ad10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 --2- 172.21.15.16:0/3549748297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3474108ef0 0x7f347419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 >> 172.21.15.16:0/3549748297 conn(0x7f34740ffe90 msgr2=0x7f34741022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 shutdown_connections 2024-05-27T06:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:52.194+0000 7f347b600700 1 -- 172.21.15.16:0/3549748297 wait complete. 2024-05-27T06:58:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T06:58:53.172 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-05-27T06:58:50.207441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665350,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1362,"num_read_kb":791,"num_write":388,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":821,"ondisk_log_size":821,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384348,"kb_used_data":14396,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557476,"statfs":{"total":1151940427776,"available":1151546855424,"internally_reserved":0,"allocated":14741504,"data_stored":8217488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":52,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006660"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102144+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T06:58:38.102144+0000","last_peered":"2024-05-27T06:58:38.102144+0000","last_clean":"2024-05-27T06:58:38.102144+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T06:58:38.102144+0000","last_undegraded":"2024-05-27T06:58:38.102144+0000","last_fullsized":"2024-05-27T06:58:38.102144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.095141+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T06:58:38.095141+0000","last_peered":"2024-05-27T06:58:38.095141+0000","last_clean":"2024-05-27T06:58:38.095141+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T06:58:38.095141+0000","last_undegraded":"2024-05-27T06:58:38.095141+0000","last_fullsized":"2024-05-27T06:58:38.095141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100844+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T06:58:38.100844+0000","last_peered":"2024-05-27T06:58:38.100844+0000","last_clean":"2024-05-27T06:58:38.100844+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T06:58:38.100844+0000","last_undegraded":"2024-05-27T06:58:38.100844+0000","last_fullsized":"2024-05-27T06:58:38.100844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.070535+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T06:58:38.070535+0000","last_peered":"2024-05-27T06:58:38.070535+0000","last_clean":"2024-05-27T06:58:38.070535+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T06:58:38.070535+0000","last_undegraded":"2024-05-27T06:58:38.070535+0000","last_fullsized":"2024-05-27T06:58:38.070535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.068454+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T06:58:38.068454+0000","last_peered":"2024-05-27T06:58:38.068454+0000","last_clean":"2024-05-27T06:58:38.068454+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T06:58:38.068454+0000","last_undegraded":"2024-05-27T06:58:38.068454+0000","last_fullsized":"2024-05-27T06:58:38.068454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089021+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T06:58:38.089021+0000","last_peered":"2024-05-27T06:58:38.089021+0000","last_clean":"2024-05-27T06:58:38.089021+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T06:58:38.089021+0000","last_undegraded":"2024-05-27T06:58:38.089021+0000","last_fullsized":"2024-05-27T06:58:38.089021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.073994+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T06:58:38.073994+0000","last_peered":"2024-05-27T06:58:38.073994+0000","last_clean":"2024-05-27T06:58:38.073994+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T06:58:38.073994+0000","last_undegraded":"2024-05-27T06:58:38.073994+0000","last_fullsized":"2024-05-27T06:58:38.073994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075947+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T06:58:38.075947+0000","last_peered":"2024-05-27T06:58:38.075947+0000","last_clean":"2024-05-27T06:58:38.075947+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T06:58:38.075947+0000","last_undegraded":"2024-05-27T06:58:38.075947+0000","last_fullsized":"2024-05-27T06:58:38.075947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.097622+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T06:58:38.097622+0000","last_peered":"2024-05-27T06:58:38.097622+0000","last_clean":"2024-05-27T06:58:38.097622+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T06:58:38.097622+0000","last_undegraded":"2024-05-27T06:58:38.097622+0000","last_fullsized":"2024-05-27T06:58:38.097622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089459+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T06:58:38.089459+0000","last_peered":"2024-05-27T06:58:38.089459+0000","last_clean":"2024-05-27T06:58:38.089459+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T06:58:38.089459+0000","last_undegraded":"2024-05-27T06:58:38.089459+0000","last_fullsized":"2024-05-27T06:58:38.089459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093482+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T06:58:38.093482+0000","last_peered":"2024-05-27T06:58:38.093482+0000","last_clean":"2024-05-27T06:58:38.093482+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T06:58:38.093482+0000","last_undegraded":"2024-05-27T06:58:38.093482+0000","last_fullsized":"2024-05-27T06:58:38.093482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.078588+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T06:58:38.078588+0000","last_peered":"2024-05-27T06:58:38.078588+0000","last_clean":"2024-05-27T06:58:38.078588+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T06:58:38.078588+0000","last_undegraded":"2024-05-27T06:58:38.078588+0000","last_fullsized":"2024-05-27T06:58:38.078588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089213+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T06:58:38.089213+0000","last_peered":"2024-05-27T06:58:38.089213+0000","last_clean":"2024-05-27T06:58:38.089213+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T06:58:38.089213+0000","last_undegraded":"2024-05-27T06:58:38.089213+0000","last_fullsized":"2024-05-27T06:58:38.089213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.099945+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T06:58:38.099945+0000","last_peered":"2024-05-27T06:58:38.099945+0000","last_clean":"2024-05-27T06:58:38.099945+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T06:58:38.099945+0000","last_undegraded":"2024-05-27T06:58:38.099945+0000","last_fullsized":"2024-05-27T06:58:38.099945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098628+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T06:58:38.098628+0000","last_peered":"2024-05-27T06:58:38.098628+0000","last_clean":"2024-05-27T06:58:38.098628+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T06:58:38.098628+0000","last_undegraded":"2024-05-27T06:58:38.098628+0000","last_fullsized":"2024-05-27T06:58:38.098628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092627+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T06:58:38.092627+0000","last_peered":"2024-05-27T06:58:38.092627+0000","last_clean":"2024-05-27T06:58:38.092627+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T06:58:38.092627+0000","last_undegraded":"2024-05-27T06:58:38.092627+0000","last_fullsized":"2024-05-27T06:58:38.092627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102225+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T06:58:38.102225+0000","last_peered":"2024-05-27T06:58:38.102225+0000","last_clean":"2024-05-27T06:58:38.102225+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T06:58:38.102225+0000","last_undegraded":"2024-05-27T06:58:38.102225+0000","last_fullsized":"2024-05-27T06:58:38.102225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092171+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T06:58:38.092171+0000","last_peered":"2024-05-27T06:58:38.092171+0000","last_clean":"2024-05-27T06:58:38.092171+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T06:58:38.092171+0000","last_undegraded":"2024-05-27T06:58:38.092171+0000","last_fullsized":"2024-05-27T06:58:38.092171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.063473+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T06:58:38.063473+0000","last_peered":"2024-05-27T06:58:38.063473+0000","last_clean":"2024-05-27T06:58:38.063473+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T06:58:38.063473+0000","last_undegraded":"2024-05-27T06:58:38.063473+0000","last_fullsized":"2024-05-27T06:58:38.063473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.412340+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T06:56:40.412340+0000","last_peered":"2024-05-27T06:56:40.412340+0000","last_clean":"2024-05-27T06:56:40.412340+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T06:56:40.412340+0000","last_undegraded":"2024-05-27T06:56:40.412340+0000","last_fullsized":"2024-05-27T06:56:40.412340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":182,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:44.820942+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T06:58:44.820942+0000","last_peered":"2024-05-27T06:58:44.820942+0000","last_clean":"2024-05-27T06:58:44.820942+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T06:58:44.820942+0000","last_undegraded":"2024-05-27T06:58:44.820942+0000","last_fullsized":"2024-05-27T06:58:44.820942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.070739+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T06:58:38.070739+0000","last_peered":"2024-05-27T06:58:38.070739+0000","last_clean":"2024-05-27T06:58:38.070739+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T06:58:38.070739+0000","last_undegraded":"2024-05-27T06:58:38.070739+0000","last_fullsized":"2024-05-27T06:58:38.070739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408788+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T06:56:40.408788+0000","last_peered":"2024-05-27T06:56:40.408788+0000","last_clean":"2024-05-27T06:56:40.408788+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T06:56:40.408788+0000","last_undegraded":"2024-05-27T06:56:40.408788+0000","last_fullsized":"2024-05-27T06:56:40.408788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.411651+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T06:56:40.411651+0000","last_peered":"2024-05-27T06:56:40.411651+0000","last_clean":"2024-05-27T06:56:40.411651+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T06:56:40.411651+0000","last_undegraded":"2024-05-27T06:56:40.411651+0000","last_fullsized":"2024-05-27T06:56:40.411651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102112+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T06:58:38.102112+0000","last_peered":"2024-05-27T06:58:38.102112+0000","last_clean":"2024-05-27T06:58:38.102112+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T06:58:38.102112+0000","last_undegraded":"2024-05-27T06:58:38.102112+0000","last_fullsized":"2024-05-27T06:58:38.102112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341777+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T06:56:40.341777+0000","last_peered":"2024-05-27T06:56:40.341777+0000","last_clean":"2024-05-27T06:56:40.341777+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T06:56:40.341777+0000","last_undegraded":"2024-05-27T06:56:40.341777+0000","last_fullsized":"2024-05-27T06:56:40.341777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449941+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T06:56:44.449941+0000","last_peered":"2024-05-27T06:56:44.449941+0000","last_clean":"2024-05-27T06:56:44.449941+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T06:56:44.449941+0000","last_undegraded":"2024-05-27T06:56:44.449941+0000","last_fullsized":"2024-05-27T06:56:44.449941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.097861+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T06:58:38.097861+0000","last_peered":"2024-05-27T06:58:38.097861+0000","last_clean":"2024-05-27T06:58:38.097861+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T06:58:38.097861+0000","last_undegraded":"2024-05-27T06:58:38.097861+0000","last_fullsized":"2024-05-27T06:58:38.097861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408756+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T06:56:40.408756+0000","last_peered":"2024-05-27T06:56:40.408756+0000","last_clean":"2024-05-27T06:56:40.408756+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T06:56:40.408756+0000","last_undegraded":"2024-05-27T06:56:40.408756+0000","last_fullsized":"2024-05-27T06:56:40.408756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.457154+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T06:56:44.457154+0000","last_peered":"2024-05-27T06:56:44.457154+0000","last_clean":"2024-05-27T06:56:44.457154+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T06:56:44.457154+0000","last_undegraded":"2024-05-27T06:56:44.457154+0000","last_fullsized":"2024-05-27T06:56:44.457154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098417+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T06:58:38.098417+0000","last_peered":"2024-05-27T06:58:38.098417+0000","last_clean":"2024-05-27T06:58:38.098417+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T06:58:38.098417+0000","last_undegraded":"2024-05-27T06:58:38.098417+0000","last_fullsized":"2024-05-27T06:58:38.098417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349631+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T06:56:40.349631+0000","last_peered":"2024-05-27T06:56:40.349631+0000","last_clean":"2024-05-27T06:56:40.349631+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T06:56:40.349631+0000","last_undegraded":"2024-05-27T06:56:40.349631+0000","last_fullsized":"2024-05-27T06:56:40.349631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":174,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:44.841801+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T06:58:44.841801+0000","last_peered":"2024-05-27T06:58:44.841801+0000","last_clean":"2024-05-27T06:58:44.841801+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T06:58:44.841801+0000","last_undegraded":"2024-05-27T06:58:44.841801+0000","last_fullsized":"2024-05-27T06:58:44.841801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.079131+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T06:58:38.079131+0000","last_peered":"2024-05-27T06:58:38.079131+0000","last_clean":"2024-05-27T06:58:38.079131+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T06:58:38.079131+0000","last_undegraded":"2024-05-27T06:58:38.079131+0000","last_fullsized":"2024-05-27T06:58:38.079131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349192+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T06:56:40.349192+0000","last_peered":"2024-05-27T06:56:40.349192+0000","last_clean":"2024-05-27T06:56:40.349192+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T06:56:40.349192+0000","last_undegraded":"2024-05-27T06:56:40.349192+0000","last_fullsized":"2024-05-27T06:56:40.349192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422145+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T06:56:41.422145+0000","last_peered":"2024-05-27T06:56:41.422145+0000","last_clean":"2024-05-27T06:56:41.422145+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T06:56:41.422145+0000","last_undegraded":"2024-05-27T06:56:41.422145+0000","last_fullsized":"2024-05-27T06:56:41.422145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077442+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T06:58:38.077442+0000","last_peered":"2024-05-27T06:58:38.077442+0000","last_clean":"2024-05-27T06:58:38.077442+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T06:58:38.077442+0000","last_undegraded":"2024-05-27T06:58:38.077442+0000","last_fullsized":"2024-05-27T06:58:38.077442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408179+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T06:56:40.408179+0000","last_peered":"2024-05-27T06:56:40.408179+0000","last_clean":"2024-05-27T06:56:40.408179+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T06:56:40.408179+0000","last_undegraded":"2024-05-27T06:56:40.408179+0000","last_fullsized":"2024-05-27T06:56:40.408179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.435023+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T06:56:42.435023+0000","last_peered":"2024-05-27T06:56:42.435023+0000","last_clean":"2024-05-27T06:56:42.435023+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T06:56:42.435023+0000","last_undegraded":"2024-05-27T06:56:42.435023+0000","last_fullsized":"2024-05-27T06:56:42.435023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.069349+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T06:58:38.069349+0000","last_peered":"2024-05-27T06:58:38.069349+0000","last_clean":"2024-05-27T06:58:38.069349+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T06:58:38.069349+0000","last_undegraded":"2024-05-27T06:58:38.069349+0000","last_fullsized":"2024-05-27T06:58:38.069349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399556+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T06:56:40.399556+0000","last_peered":"2024-05-27T06:56:40.399556+0000","last_clean":"2024-05-27T06:56:40.399556+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T06:56:40.399556+0000","last_undegraded":"2024-05-27T06:56:40.399556+0000","last_fullsized":"2024-05-27T06:56:40.399556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.456993+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T06:56:44.456993+0000","last_peered":"2024-05-27T06:56:44.456993+0000","last_clean":"2024-05-27T06:56:44.456993+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T06:56:44.456993+0000","last_undegraded":"2024-05-27T06:56:44.456993+0000","last_fullsized":"2024-05-27T06:56:44.456993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.083741+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T06:58:38.083741+0000","last_peered":"2024-05-27T06:58:38.083741+0000","last_clean":"2024-05-27T06:58:38.083741+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T06:58:38.083741+0000","last_undegraded":"2024-05-27T06:58:38.083741+0000","last_fullsized":"2024-05-27T06:58:38.083741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402777+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T06:56:40.402777+0000","last_peered":"2024-05-27T06:56:40.402777+0000","last_clean":"2024-05-27T06:56:40.402777+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T06:56:40.402777+0000","last_undegraded":"2024-05-27T06:56:40.402777+0000","last_fullsized":"2024-05-27T06:56:40.402777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:46.861990+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T06:58:46.861990+0000","last_peered":"2024-05-27T06:58:46.861990+0000","last_clean":"2024-05-27T06:58:46.861990+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T06:58:46.861990+0000","last_undegraded":"2024-05-27T06:58:46.861990+0000","last_fullsized":"2024-05-27T06:58:46.861990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094922+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T06:58:38.094922+0000","last_peered":"2024-05-27T06:58:38.094922+0000","last_clean":"2024-05-27T06:58:38.094922+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T06:58:38.094922+0000","last_undegraded":"2024-05-27T06:58:38.094922+0000","last_fullsized":"2024-05-27T06:58:38.094922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398619+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T06:56:40.398619+0000","last_peered":"2024-05-27T06:56:40.398619+0000","last_clean":"2024-05-27T06:56:40.398619+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T06:56:40.398619+0000","last_undegraded":"2024-05-27T06:56:40.398619+0000","last_fullsized":"2024-05-27T06:56:40.398619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434594+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T06:56:42.434594+0000","last_peered":"2024-05-27T06:56:42.434594+0000","last_clean":"2024-05-27T06:56:42.434594+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T06:56:42.434594+0000","last_undegraded":"2024-05-27T06:56:42.434594+0000","last_fullsized":"2024-05-27T06:56:42.434594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100225+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T06:58:38.100225+0000","last_peered":"2024-05-27T06:58:38.100225+0000","last_clean":"2024-05-27T06:58:38.100225+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T06:58:38.100225+0000","last_undegraded":"2024-05-27T06:58:38.100225+0000","last_fullsized":"2024-05-27T06:58:38.100225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342480+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T06:56:40.342480+0000","last_peered":"2024-05-27T06:56:40.342480+0000","last_clean":"2024-05-27T06:56:40.342480+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T06:56:40.342480+0000","last_undegraded":"2024-05-27T06:56:40.342480+0000","last_fullsized":"2024-05-27T06:56:40.342480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.456185+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T06:56:44.456185+0000","last_peered":"2024-05-27T06:56:44.456185+0000","last_clean":"2024-05-27T06:56:44.456185+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T06:56:44.456185+0000","last_undegraded":"2024-05-27T06:56:44.456185+0000","last_fullsized":"2024-05-27T06:56:44.456185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098075+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T06:58:38.098075+0000","last_peered":"2024-05-27T06:58:38.098075+0000","last_clean":"2024-05-27T06:58:38.098075+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T06:58:38.098075+0000","last_undegraded":"2024-05-27T06:58:38.098075+0000","last_fullsized":"2024-05-27T06:58:38.098075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399820+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T06:56:40.399820+0000","last_peered":"2024-05-27T06:56:40.399820+0000","last_clean":"2024-05-27T06:56:40.399820+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T06:56:40.399820+0000","last_undegraded":"2024-05-27T06:56:40.399820+0000","last_fullsized":"2024-05-27T06:56:40.399820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'39","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:46.857418+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T06:58:46.857418+0000","last_peered":"2024-05-27T06:58:46.857418+0000","last_clean":"2024-05-27T06:58:46.857418+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T06:58:46.857418+0000","last_undegraded":"2024-05-27T06:58:46.857418+0000","last_fullsized":"2024-05-27T06:58:46.857418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66730,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":258,"num_read_kb":194,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.074184+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T06:58:38.074184+0000","last_peered":"2024-05-27T06:58:38.074184+0000","last_clean":"2024-05-27T06:58:38.074184+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T06:58:38.074184+0000","last_undegraded":"2024-05-27T06:58:38.074184+0000","last_fullsized":"2024-05-27T06:58:38.074184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342280+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T06:56:40.342280+0000","last_peered":"2024-05-27T06:56:40.342280+0000","last_clean":"2024-05-27T06:56:40.342280+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T06:56:40.342280+0000","last_undegraded":"2024-05-27T06:56:40.342280+0000","last_fullsized":"2024-05-27T06:56:40.342280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451259+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T06:56:44.451259+0000","last_peered":"2024-05-27T06:56:44.451259+0000","last_clean":"2024-05-27T06:56:44.451259+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T06:56:44.451259+0000","last_undegraded":"2024-05-27T06:56:44.451259+0000","last_fullsized":"2024-05-27T06:56:44.451259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.083869+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T06:58:38.083869+0000","last_peered":"2024-05-27T06:58:38.083869+0000","last_clean":"2024-05-27T06:58:38.083869+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T06:58:38.083869+0000","last_undegraded":"2024-05-27T06:58:38.083869+0000","last_fullsized":"2024-05-27T06:58:38.083869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401558+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T06:56:40.401558+0000","last_peered":"2024-05-27T06:56:40.401558+0000","last_clean":"2024-05-27T06:56:40.401558+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T06:56:40.401558+0000","last_undegraded":"2024-05-27T06:56:40.401558+0000","last_fullsized":"2024-05-27T06:56:40.401558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449358+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T06:56:44.449358+0000","last_peered":"2024-05-27T06:56:44.449358+0000","last_clean":"2024-05-27T06:56:44.449358+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T06:56:44.449358+0000","last_undegraded":"2024-05-27T06:56:44.449358+0000","last_fullsized":"2024-05-27T06:56:44.449358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094375+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T06:58:38.094375+0000","last_peered":"2024-05-27T06:58:38.094375+0000","last_clean":"2024-05-27T06:58:38.094375+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T06:58:38.094375+0000","last_undegraded":"2024-05-27T06:58:38.094375+0000","last_fullsized":"2024-05-27T06:58:38.094375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402173+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T06:56:40.402173+0000","last_peered":"2024-05-27T06:56:40.402173+0000","last_clean":"2024-05-27T06:56:40.402173+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T06:56:40.402173+0000","last_undegraded":"2024-05-27T06:56:40.402173+0000","last_fullsized":"2024-05-27T06:56:40.402173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:45.855857+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T06:58:45.855857+0000","last_peered":"2024-05-27T06:58:45.855857+0000","last_clean":"2024-05-27T06:58:45.855857+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T06:58:45.855857+0000","last_undegraded":"2024-05-27T06:58:45.855857+0000","last_fullsized":"2024-05-27T06:58:45.855857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.082881+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T06:58:38.082881+0000","last_peered":"2024-05-27T06:58:38.082881+0000","last_clean":"2024-05-27T06:58:38.082881+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T06:58:38.082881+0000","last_undegraded":"2024-05-27T06:58:38.082881+0000","last_fullsized":"2024-05-27T06:58:38.082881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335420+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T06:56:40.335420+0000","last_peered":"2024-05-27T06:56:40.335420+0000","last_clean":"2024-05-27T06:56:40.335420+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T06:56:40.335420+0000","last_undegraded":"2024-05-27T06:56:40.335420+0000","last_fullsized":"2024-05-27T06:56:40.335420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449699+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T06:56:44.449699+0000","last_peered":"2024-05-27T06:56:44.449699+0000","last_clean":"2024-05-27T06:56:44.449699+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T06:56:44.449699+0000","last_undegraded":"2024-05-27T06:56:44.449699+0000","last_fullsized":"2024-05-27T06:56:44.449699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096423+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T06:58:38.096423+0000","last_peered":"2024-05-27T06:58:38.096423+0000","last_clean":"2024-05-27T06:58:38.096423+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T06:58:38.096423+0000","last_undegraded":"2024-05-27T06:58:38.096423+0000","last_fullsized":"2024-05-27T06:58:38.096423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340204+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T06:56:40.340204+0000","last_peered":"2024-05-27T06:56:40.340204+0000","last_clean":"2024-05-27T06:56:40.340204+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T06:56:40.340204+0000","last_undegraded":"2024-05-27T06:56:40.340204+0000","last_fullsized":"2024-05-27T06:56:40.340204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451292+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T06:56:44.451292+0000","last_peered":"2024-05-27T06:56:44.451292+0000","last_clean":"2024-05-27T06:56:44.451292+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T06:56:44.451292+0000","last_undegraded":"2024-05-27T06:56:44.451292+0000","last_fullsized":"2024-05-27T06:56:44.451292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075829+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T06:58:38.075829+0000","last_peered":"2024-05-27T06:58:38.075829+0000","last_clean":"2024-05-27T06:58:38.075829+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T06:58:38.075829+0000","last_undegraded":"2024-05-27T06:58:38.075829+0000","last_fullsized":"2024-05-27T06:58:38.075829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401521+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T06:56:40.401521+0000","last_peered":"2024-05-27T06:56:40.401521+0000","last_clean":"2024-05-27T06:56:40.401521+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T06:56:40.401521+0000","last_undegraded":"2024-05-27T06:56:40.401521+0000","last_fullsized":"2024-05-27T06:56:40.401521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422923+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T06:56:41.422923+0000","last_peered":"2024-05-27T06:56:41.422923+0000","last_clean":"2024-05-27T06:56:41.422923+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T06:56:41.422923+0000","last_undegraded":"2024-05-27T06:56:41.422923+0000","last_fullsized":"2024-05-27T06:56:41.422923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087790+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T06:58:38.087790+0000","last_peered":"2024-05-27T06:58:38.087790+0000","last_clean":"2024-05-27T06:58:38.087790+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T06:58:38.087790+0000","last_undegraded":"2024-05-27T06:58:38.087790+0000","last_fullsized":"2024-05-27T06:58:38.087790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335922+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T06:56:40.335922+0000","last_peered":"2024-05-27T06:56:40.335922+0000","last_clean":"2024-05-27T06:56:40.335922+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T06:56:40.335922+0000","last_undegraded":"2024-05-27T06:56:40.335922+0000","last_fullsized":"2024-05-27T06:56:40.335922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":180,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:47.830866+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T06:58:47.830866+0000","last_peered":"2024-05-27T06:58:47.830866+0000","last_clean":"2024-05-27T06:58:47.830866+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T06:58:47.830866+0000","last_undegraded":"2024-05-27T06:58:47.830866+0000","last_fullsized":"2024-05-27T06:58:47.830866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.090877+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T06:58:38.090877+0000","last_peered":"2024-05-27T06:58:38.090877+0000","last_clean":"2024-05-27T06:58:38.090877+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T06:58:38.090877+0000","last_undegraded":"2024-05-27T06:58:38.090877+0000","last_fullsized":"2024-05-27T06:58:38.090877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335077+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T06:56:40.335077+0000","last_peered":"2024-05-27T06:56:40.335077+0000","last_clean":"2024-05-27T06:56:40.335077+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T06:56:40.335077+0000","last_undegraded":"2024-05-27T06:56:40.335077+0000","last_fullsized":"2024-05-27T06:56:40.335077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076770+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T06:58:38.076770+0000","last_peered":"2024-05-27T06:58:38.076770+0000","last_clean":"2024-05-27T06:58:38.076770+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T06:58:38.076770+0000","last_undegraded":"2024-05-27T06:58:38.076770+0000","last_fullsized":"2024-05-27T06:58:38.076770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:45.854173+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T06:58:45.854173+0000","last_peered":"2024-05-27T06:58:45.854173+0000","last_clean":"2024-05-27T06:58:45.854173+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T06:58:45.854173+0000","last_undegraded":"2024-05-27T06:58:45.854173+0000","last_fullsized":"2024-05-27T06:58:45.854173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:57.726620+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T06:56:57.726620+0000","last_peered":"2024-05-27T06:56:57.726620+0000","last_clean":"2024-05-27T06:56:57.726620+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T06:56:57.726620+0000","last_undegraded":"2024-05-27T06:56:57.726620+0000","last_fullsized":"2024-05-27T06:56:57.726620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092774+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T06:58:38.092774+0000","last_peered":"2024-05-27T06:58:38.092774+0000","last_clean":"2024-05-27T06:58:38.092774+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T06:58:38.092774+0000","last_undegraded":"2024-05-27T06:58:38.092774+0000","last_fullsized":"2024-05-27T06:58:38.092774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450805+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T06:56:44.450805+0000","last_peered":"2024-05-27T06:56:44.450805+0000","last_clean":"2024-05-27T06:56:44.450805+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T06:56:44.450805+0000","last_undegraded":"2024-05-27T06:56:44.450805+0000","last_fullsized":"2024-05-27T06:56:44.450805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.337815+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T06:56:40.337815+0000","last_peered":"2024-05-27T06:56:40.337815+0000","last_clean":"2024-05-27T06:56:40.337815+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T06:56:40.337815+0000","last_undegraded":"2024-05-27T06:56:40.337815+0000","last_fullsized":"2024-05-27T06:56:40.337815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342212+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T06:56:40.342212+0000","last_peered":"2024-05-27T06:56:40.342212+0000","last_clean":"2024-05-27T06:56:40.342212+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T06:56:40.342212+0000","last_undegraded":"2024-05-27T06:56:40.342212+0000","last_fullsized":"2024-05-27T06:56:40.342212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077957+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T06:58:38.077957+0000","last_peered":"2024-05-27T06:58:38.077957+0000","last_clean":"2024-05-27T06:58:38.077957+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T06:58:38.077957+0000","last_undegraded":"2024-05-27T06:58:38.077957+0000","last_fullsized":"2024-05-27T06:58:38.077957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432537+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T06:56:42.432537+0000","last_peered":"2024-05-27T06:56:42.432537+0000","last_clean":"2024-05-27T06:56:42.432537+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T06:56:42.432537+0000","last_undegraded":"2024-05-27T06:56:42.432537+0000","last_fullsized":"2024-05-27T06:56:42.432537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402603+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T06:56:40.402603+0000","last_peered":"2024-05-27T06:56:40.402603+0000","last_clean":"2024-05-27T06:56:40.402603+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T06:56:40.402603+0000","last_undegraded":"2024-05-27T06:56:40.402603+0000","last_fullsized":"2024-05-27T06:56:40.402603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450385+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T06:56:44.450385+0000","last_peered":"2024-05-27T06:56:44.450385+0000","last_clean":"2024-05-27T06:56:44.450385+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T06:56:44.450385+0000","last_undegraded":"2024-05-27T06:56:44.450385+0000","last_fullsized":"2024-05-27T06:56:44.450385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.078740+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T06:58:38.078740+0000","last_peered":"2024-05-27T06:58:38.078740+0000","last_clean":"2024-05-27T06:58:38.078740+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T06:58:38.078740+0000","last_undegraded":"2024-05-27T06:58:38.078740+0000","last_fullsized":"2024-05-27T06:58:38.078740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404013+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T06:56:40.404013+0000","last_peered":"2024-05-27T06:56:40.404013+0000","last_clean":"2024-05-27T06:56:40.404013+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T06:56:40.404013+0000","last_undegraded":"2024-05-27T06:56:40.404013+0000","last_fullsized":"2024-05-27T06:56:40.404013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'17","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:17.444934+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T06:57:17.444934+0000","last_peered":"2024-05-27T06:57:17.444934+0000","last_clean":"2024-05-27T06:57:17.444934+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T06:57:17.444934+0000","last_undegraded":"2024-05-27T06:57:17.444934+0000","last_fullsized":"2024-05-27T06:57:17.444934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087132+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T06:58:38.087132+0000","last_peered":"2024-05-27T06:58:38.087132+0000","last_clean":"2024-05-27T06:58:38.087132+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T06:58:38.087132+0000","last_undegraded":"2024-05-27T06:58:38.087132+0000","last_fullsized":"2024-05-27T06:58:38.087132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348360+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T06:56:40.348360+0000","last_peered":"2024-05-27T06:56:40.348360+0000","last_clean":"2024-05-27T06:56:40.348360+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T06:56:40.348360+0000","last_undegraded":"2024-05-27T06:56:40.348360+0000","last_fullsized":"2024-05-27T06:56:40.348360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449227+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T06:56:44.449227+0000","last_peered":"2024-05-27T06:56:44.449227+0000","last_clean":"2024-05-27T06:56:44.449227+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T06:56:44.449227+0000","last_undegraded":"2024-05-27T06:56:44.449227+0000","last_fullsized":"2024-05-27T06:56:44.449227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094673+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T06:58:38.094673+0000","last_peered":"2024-05-27T06:58:38.094673+0000","last_clean":"2024-05-27T06:58:38.094673+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T06:58:38.094673+0000","last_undegraded":"2024-05-27T06:58:38.094673+0000","last_fullsized":"2024-05-27T06:58:38.094673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344207+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T06:56:40.344207+0000","last_peered":"2024-05-27T06:56:40.344207+0000","last_clean":"2024-05-27T06:56:40.344207+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T06:56:40.344207+0000","last_undegraded":"2024-05-27T06:56:40.344207+0000","last_fullsized":"2024-05-27T06:56:40.344207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394827+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T06:56:40.394827+0000","last_peered":"2024-05-27T06:56:40.394827+0000","last_clean":"2024-05-27T06:56:40.394827+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T06:56:40.394827+0000","last_undegraded":"2024-05-27T06:56:40.394827+0000","last_fullsized":"2024-05-27T06:56:40.394827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.025373+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T06:58:38.025373+0000","last_peered":"2024-05-27T06:58:38.025373+0000","last_clean":"2024-05-27T06:58:38.025373+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T06:58:38.025373+0000","last_undegraded":"2024-05-27T06:58:38.025373+0000","last_fullsized":"2024-05-27T06:58:38.025373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402580+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T06:56:40.402580+0000","last_peered":"2024-05-27T06:56:40.402580+0000","last_clean":"2024-05-27T06:56:40.402580+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T06:56:40.402580+0000","last_undegraded":"2024-05-27T06:56:40.402580+0000","last_fullsized":"2024-05-27T06:56:40.402580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434757+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T06:56:42.434757+0000","last_peered":"2024-05-27T06:56:42.434757+0000","last_clean":"2024-05-27T06:56:42.434757+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T06:56:42.434757+0000","last_undegraded":"2024-05-27T06:56:42.434757+0000","last_fullsized":"2024-05-27T06:56:42.434757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100321+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T06:58:38.100321+0000","last_peered":"2024-05-27T06:58:38.100321+0000","last_clean":"2024-05-27T06:58:38.100321+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T06:58:38.100321+0000","last_undegraded":"2024-05-27T06:58:38.100321+0000","last_fullsized":"2024-05-27T06:58:38.100321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:06.295632+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T06:57:06.295632+0000","last_peered":"2024-05-27T06:57:06.295632+0000","last_clean":"2024-05-27T06:57:06.295632+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T06:57:06.295632+0000","last_undegraded":"2024-05-27T06:57:06.295632+0000","last_fullsized":"2024-05-27T06:57:06.295632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.407953+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T06:56:42.407953+0000","last_peered":"2024-05-27T06:56:42.407953+0000","last_clean":"2024-05-27T06:56:42.407953+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T06:56:42.407953+0000","last_undegraded":"2024-05-27T06:56:42.407953+0000","last_fullsized":"2024-05-27T06:56:42.407953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.090494+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T06:58:38.090494+0000","last_peered":"2024-05-27T06:58:38.090494+0000","last_clean":"2024-05-27T06:58:38.090494+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T06:58:38.090494+0000","last_undegraded":"2024-05-27T06:58:38.090494+0000","last_fullsized":"2024-05-27T06:58:38.090494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344102+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T06:56:40.344102+0000","last_peered":"2024-05-27T06:56:40.344102+0000","last_clean":"2024-05-27T06:56:40.344102+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T06:56:40.344102+0000","last_undegraded":"2024-05-27T06:56:40.344102+0000","last_fullsized":"2024-05-27T06:56:40.344102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451041+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T06:56:44.451041+0000","last_peered":"2024-05-27T06:56:44.451041+0000","last_clean":"2024-05-27T06:56:44.451041+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T06:56:44.451041+0000","last_undegraded":"2024-05-27T06:56:44.451041+0000","last_fullsized":"2024-05-27T06:56:44.451041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093437+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T06:58:38.093437+0000","last_peered":"2024-05-27T06:58:38.093437+0000","last_clean":"2024-05-27T06:58:38.093437+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T06:58:38.093437+0000","last_undegraded":"2024-05-27T06:58:38.093437+0000","last_fullsized":"2024-05-27T06:58:38.093437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396247+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T06:56:40.396247+0000","last_peered":"2024-05-27T06:56:40.396247+0000","last_clean":"2024-05-27T06:56:40.396247+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T06:56:40.396247+0000","last_undegraded":"2024-05-27T06:56:40.396247+0000","last_fullsized":"2024-05-27T06:56:40.396247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":181,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:47.866511+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T06:58:47.866511+0000","last_peered":"2024-05-27T06:58:47.866511+0000","last_clean":"2024-05-27T06:58:47.866511+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T06:58:47.866511+0000","last_undegraded":"2024-05-27T06:58:47.866511+0000","last_fullsized":"2024-05-27T06:58:47.866511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093626+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T06:58:38.093626+0000","last_peered":"2024-05-27T06:58:38.093626+0000","last_clean":"2024-05-27T06:58:38.093626+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T06:58:38.093626+0000","last_undegraded":"2024-05-27T06:58:38.093626+0000","last_fullsized":"2024-05-27T06:58:38.093626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401123+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T06:56:40.401123+0000","last_peered":"2024-05-27T06:56:40.401123+0000","last_clean":"2024-05-27T06:56:40.401123+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T06:56:40.401123+0000","last_undegraded":"2024-05-27T06:56:40.401123+0000","last_fullsized":"2024-05-27T06:56:40.401123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451295+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T06:56:44.451295+0000","last_peered":"2024-05-27T06:56:44.451295+0000","last_clean":"2024-05-27T06:56:44.451295+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T06:56:44.451295+0000","last_undegraded":"2024-05-27T06:56:44.451295+0000","last_fullsized":"2024-05-27T06:56:44.451295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.073707+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T06:58:38.073707+0000","last_peered":"2024-05-27T06:58:38.073707+0000","last_clean":"2024-05-27T06:58:38.073707+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T06:58:38.073707+0000","last_undegraded":"2024-05-27T06:58:38.073707+0000","last_fullsized":"2024-05-27T06:58:38.073707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339817+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T06:56:40.339817+0000","last_peered":"2024-05-27T06:56:40.339817+0000","last_clean":"2024-05-27T06:56:40.339817+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T06:56:40.339817+0000","last_undegraded":"2024-05-27T06:56:40.339817+0000","last_fullsized":"2024-05-27T06:56:40.339817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432981+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T06:56:42.432981+0000","last_peered":"2024-05-27T06:56:42.432981+0000","last_clean":"2024-05-27T06:56:42.432981+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T06:56:42.432981+0000","last_undegraded":"2024-05-27T06:56:42.432981+0000","last_fullsized":"2024-05-27T06:56:42.432981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089291+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T06:58:38.089291+0000","last_peered":"2024-05-27T06:58:38.089291+0000","last_clean":"2024-05-27T06:58:38.089291+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T06:58:38.089291+0000","last_undegraded":"2024-05-27T06:58:38.089291+0000","last_fullsized":"2024-05-27T06:58:38.089291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076863+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T06:58:38.076863+0000","last_peered":"2024-05-27T06:58:38.076863+0000","last_clean":"2024-05-27T06:58:38.076863+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T06:58:38.076863+0000","last_undegraded":"2024-05-27T06:58:38.076863+0000","last_fullsized":"2024-05-27T06:58:38.076863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075225+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T06:58:38.075225+0000","last_peered":"2024-05-27T06:58:38.075225+0000","last_clean":"2024-05-27T06:58:38.075225+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T06:58:38.075225+0000","last_undegraded":"2024-05-27T06:58:38.075225+0000","last_fullsized":"2024-05-27T06:58:38.075225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.101321+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T06:58:38.101321+0000","last_peered":"2024-05-27T06:58:38.101321+0000","last_clean":"2024-05-27T06:58:38.101321+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T06:58:38.101321+0000","last_undegraded":"2024-05-27T06:58:38.101321+0000","last_fullsized":"2024-05-27T06:58:38.101321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098334+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T06:58:38.098334+0000","last_peered":"2024-05-27T06:58:38.098334+0000","last_clean":"2024-05-27T06:58:38.098334+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T06:58:38.098334+0000","last_undegraded":"2024-05-27T06:58:38.098334+0000","last_fullsized":"2024-05-27T06:58:38.098334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096554+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T06:58:38.096554+0000","last_peered":"2024-05-27T06:58:38.096554+0000","last_clean":"2024-05-27T06:58:38.096554+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T06:58:38.096554+0000","last_undegraded":"2024-05-27T06:58:38.096554+0000","last_fullsized":"2024-05-27T06:58:38.096554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076044+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T06:58:38.076044+0000","last_peered":"2024-05-27T06:58:38.076044+0000","last_clean":"2024-05-27T06:58:38.076044+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T06:58:38.076044+0000","last_undegraded":"2024-05-27T06:58:38.076044+0000","last_fullsized":"2024-05-27T06:58:38.076044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087261+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T06:58:38.087261+0000","last_peered":"2024-05-27T06:58:38.087261+0000","last_clean":"2024-05-27T06:58:38.087261+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T06:58:38.087261+0000","last_undegraded":"2024-05-27T06:58:38.087261+0000","last_fullsized":"2024-05-27T06:58:38.087261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.066236+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T06:58:38.066236+0000","last_peered":"2024-05-27T06:58:38.066236+0000","last_clean":"2024-05-27T06:58:38.066236+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T06:58:38.066236+0000","last_undegraded":"2024-05-27T06:58:38.066236+0000","last_fullsized":"2024-05-27T06:58:38.066236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.080408+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T06:58:38.080408+0000","last_peered":"2024-05-27T06:58:38.080408+0000","last_clean":"2024-05-27T06:58:38.080408+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T06:58:38.080408+0000","last_undegraded":"2024-05-27T06:58:38.080408+0000","last_fullsized":"2024-05-27T06:58:38.080408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.067486+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T06:58:38.067486+0000","last_peered":"2024-05-27T06:58:38.067486+0000","last_clean":"2024-05-27T06:58:38.067486+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T06:58:38.067486+0000","last_undegraded":"2024-05-27T06:58:38.067486+0000","last_fullsized":"2024-05-27T06:58:38.067486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094527+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T06:58:38.094527+0000","last_peered":"2024-05-27T06:58:38.094527+0000","last_clean":"2024-05-27T06:58:38.094527+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T06:58:38.094527+0000","last_undegraded":"2024-05-27T06:58:38.094527+0000","last_fullsized":"2024-05-27T06:58:38.094527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096850+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T06:58:38.096850+0000","last_peered":"2024-05-27T06:58:38.096850+0000","last_clean":"2024-05-27T06:58:38.096850+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T06:58:38.096850+0000","last_undegraded":"2024-05-27T06:58:38.096850+0000","last_fullsized":"2024-05-27T06:58:38.096850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077760+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T06:58:38.077760+0000","last_peered":"2024-05-27T06:58:38.077760+0000","last_clean":"2024-05-27T06:58:38.077760+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T06:58:38.077760+0000","last_undegraded":"2024-05-27T06:58:38.077760+0000","last_fullsized":"2024-05-27T06:58:38.077760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74748,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1281,"num_read_kb":722,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":224244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678081,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874312,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1175147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037840,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":521377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430304,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1662976,"data_stored":1118547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626534,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738514,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":518491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182524,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986293,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":587839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822766,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T06:58:53.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T06:58:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:53 smithi069 ceph-mon[35276]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:53.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:53 smithi069 ceph-mon[35276]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-27T06:58:53.635 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:53 smithi164 ceph-mon[35327]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:53.635 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:53 smithi164 ceph-mon[35327]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-27T06:58:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:53.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:53 smithi016 ceph-mon[31415]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:53.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:53 smithi016 ceph-mon[31415]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-27T06:58:55.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:55 smithi016 ceph-mon[31415]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:58:55.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:55 smithi164 ceph-mon[35327]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:58:55.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:55 smithi069 ceph-mon[35276]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-27T06:58:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 -- 172.21.15.16:0/1603272904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410a980 msgr2=0x7f6e3410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 --2- 172.21.15.16:0/1603272904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410a980 0x7f6e3410ad90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009990 tx=0x7f6e1c009ca0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 -- 172.21.15.16:0/1603272904 shutdown_connections 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 --2- 172.21.15.16:0/1603272904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e3410b2d0 0x7f6e34117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 --2- 172.21.15.16:0/1603272904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e34110960 0x7f6e34110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 --2- 172.21.15.16:0/1603272904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410a980 0x7f6e3410ad90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.188+0000 7f6e3a600700 1 -- 172.21.15.16:0/1603272904 >> 172.21.15.16:0/1603272904 conn(0x7f6e34077c70 msgr2=0x7f6e340780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.189+0000 7f6e3a600700 1 -- 172.21.15.16:0/1603272904 shutdown_connections 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.189+0000 7f6e3a600700 1 -- 172.21.15.16:0/1603272904 wait complete. 2024-05-27T06:58:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.190+0000 7f6e3a600700 1 Processor -- start 2024-05-27T06:58:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.190+0000 7f6e3a600700 1 -- start start 2024-05-27T06:58:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3410a980 0x7f6e341a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 0x7f6e341a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:56.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3411a980 con 0x7f6e3410b2d0 2024-05-27T06:58:56.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3411a800 con 0x7f6e34110960 2024-05-27T06:58:56.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e3a600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3411ab00 con 0x7f6e3410a980 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.191+0000 7f6e2aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41514/0 (socket says 172.21.15.16:41514) 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 -- 172.21.15.16:0/4137493636 learned_addr learned my addr 172.21.15.16:0/4137493636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e33400700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3410a980 0x7f6e341a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e33e00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 0x7f6e341a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 -- 172.21.15.16:0/4137493636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3410a980 msgr2=0x7f6e341a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3410a980 0x7f6e341a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 -- 172.21.15.16:0/4137493636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 msgr2=0x7f6e341a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 0x7f6e341a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e2aa00700 1 -- 172.21.15.16:0/4137493636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e341a6280 con 0x7f6e3410b2d0 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.192+0000 7f6e33e00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 0x7f6e341a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:58:56.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e2aa00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:56.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c027070 con 0x7f6e3410b2d0 2024-05-27T06:58:56.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e341a6510 con 0x7f6e3410b2d0 2024-05-27T06:58:56.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e1c00fb00 con 0x7f6e3410b2d0 2024-05-27T06:58:56.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c022b50 con 0x7f6e3410b2d0 2024-05-27T06:58:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.193+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341ade90 con 0x7f6e3410b2d0 2024-05-27T06:58:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.195+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3410c070 con 0x7f6e3410b2d0 2024-05-27T06:58:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.197+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6e1c02a070 con 0x7f6e3410b2d0 2024-05-27T06:58:56.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.199+0000 7f6e30c00700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 0x7f6e180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:58:56.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.199+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6e1c0e3890 con 0x7f6e3410b2d0 2024-05-27T06:58:56.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.199+0000 7f6e33400700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 0x7f6e180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:58:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.200+0000 7f6e33400700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 0x7f6e180b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005950 tx=0x7f6e2400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:58:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.201+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e1c05fdf0 con 0x7f6e3410b2d0 2024-05-27T06:58:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.443+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e341a2710 con 0x7f6e180b2880 2024-05-27T06:58:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.474+0000 7f6e30c00700 1 -- 172.21.15.16:0/4137493636 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340530 (secure 0 0 0) 0x7f6e341a2710 con 0x7f6e180b2880 2024-05-27T06:58:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:58:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.484+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 msgr2=0x7f6e180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.484+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 0x7f6e180b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005950 tx=0x7f6e2400b410 comp rx=0 tx=0).stop 2024-05-27T06:58:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 msgr2=0x7f6e341a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:58:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c0057a0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 shutdown_connections 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f6e180b2880 0x7f6e180b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e34110960 0x7f6e341a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3410a980 0x7f6e341a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 --2- 172.21.15.16:0/4137493636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3410b2d0 0x7f6e341a1950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 >> 172.21.15.16:0/4137493636 conn(0x7f6e34077c70 msgr2=0x7f6e3407b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 shutdown_connections 2024-05-27T06:58:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:58:56.485+0000 7f6e3a600700 1 -- 172.21.15.16:0/4137493636 wait complete. 2024-05-27T06:58:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T06:58:57.475 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-05-27T06:58:56.210709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665350,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1411,"num_read_kb":815,"num_write":388,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":821,"ondisk_log_size":821,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384348,"kb_used_data":14396,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557476,"statfs":{"total":1151940427776,"available":1151546855424,"internally_reserved":0,"allocated":14741504,"data_stored":8217488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":47,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006918"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102144+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T06:58:38.102144+0000","last_peered":"2024-05-27T06:58:38.102144+0000","last_clean":"2024-05-27T06:58:38.102144+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T06:58:38.102144+0000","last_undegraded":"2024-05-27T06:58:38.102144+0000","last_fullsized":"2024-05-27T06:58:38.102144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.095141+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T06:58:38.095141+0000","last_peered":"2024-05-27T06:58:38.095141+0000","last_clean":"2024-05-27T06:58:38.095141+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T06:58:38.095141+0000","last_undegraded":"2024-05-27T06:58:38.095141+0000","last_fullsized":"2024-05-27T06:58:38.095141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100844+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T06:58:38.100844+0000","last_peered":"2024-05-27T06:58:38.100844+0000","last_clean":"2024-05-27T06:58:38.100844+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T06:58:38.100844+0000","last_undegraded":"2024-05-27T06:58:38.100844+0000","last_fullsized":"2024-05-27T06:58:38.100844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.070535+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T06:58:38.070535+0000","last_peered":"2024-05-27T06:58:38.070535+0000","last_clean":"2024-05-27T06:58:38.070535+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T06:58:38.070535+0000","last_undegraded":"2024-05-27T06:58:38.070535+0000","last_fullsized":"2024-05-27T06:58:38.070535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.068454+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T06:58:38.068454+0000","last_peered":"2024-05-27T06:58:38.068454+0000","last_clean":"2024-05-27T06:58:38.068454+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T06:58:38.068454+0000","last_undegraded":"2024-05-27T06:58:38.068454+0000","last_fullsized":"2024-05-27T06:58:38.068454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089021+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T06:58:38.089021+0000","last_peered":"2024-05-27T06:58:38.089021+0000","last_clean":"2024-05-27T06:58:38.089021+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T06:58:38.089021+0000","last_undegraded":"2024-05-27T06:58:38.089021+0000","last_fullsized":"2024-05-27T06:58:38.089021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.073994+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T06:58:38.073994+0000","last_peered":"2024-05-27T06:58:38.073994+0000","last_clean":"2024-05-27T06:58:38.073994+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T06:58:38.073994+0000","last_undegraded":"2024-05-27T06:58:38.073994+0000","last_fullsized":"2024-05-27T06:58:38.073994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075947+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T06:58:38.075947+0000","last_peered":"2024-05-27T06:58:38.075947+0000","last_clean":"2024-05-27T06:58:38.075947+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T06:58:38.075947+0000","last_undegraded":"2024-05-27T06:58:38.075947+0000","last_fullsized":"2024-05-27T06:58:38.075947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.097622+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T06:58:38.097622+0000","last_peered":"2024-05-27T06:58:38.097622+0000","last_clean":"2024-05-27T06:58:38.097622+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T06:58:38.097622+0000","last_undegraded":"2024-05-27T06:58:38.097622+0000","last_fullsized":"2024-05-27T06:58:38.097622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089459+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T06:58:38.089459+0000","last_peered":"2024-05-27T06:58:38.089459+0000","last_clean":"2024-05-27T06:58:38.089459+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T06:58:38.089459+0000","last_undegraded":"2024-05-27T06:58:38.089459+0000","last_fullsized":"2024-05-27T06:58:38.089459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093482+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T06:58:38.093482+0000","last_peered":"2024-05-27T06:58:38.093482+0000","last_clean":"2024-05-27T06:58:38.093482+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T06:58:38.093482+0000","last_undegraded":"2024-05-27T06:58:38.093482+0000","last_fullsized":"2024-05-27T06:58:38.093482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.078588+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T06:58:38.078588+0000","last_peered":"2024-05-27T06:58:38.078588+0000","last_clean":"2024-05-27T06:58:38.078588+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T06:58:38.078588+0000","last_undegraded":"2024-05-27T06:58:38.078588+0000","last_fullsized":"2024-05-27T06:58:38.078588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089213+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T06:58:38.089213+0000","last_peered":"2024-05-27T06:58:38.089213+0000","last_clean":"2024-05-27T06:58:38.089213+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T06:58:38.089213+0000","last_undegraded":"2024-05-27T06:58:38.089213+0000","last_fullsized":"2024-05-27T06:58:38.089213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.099945+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T06:58:38.099945+0000","last_peered":"2024-05-27T06:58:38.099945+0000","last_clean":"2024-05-27T06:58:38.099945+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T06:58:38.099945+0000","last_undegraded":"2024-05-27T06:58:38.099945+0000","last_fullsized":"2024-05-27T06:58:38.099945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098628+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T06:58:38.098628+0000","last_peered":"2024-05-27T06:58:38.098628+0000","last_clean":"2024-05-27T06:58:38.098628+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T06:58:38.098628+0000","last_undegraded":"2024-05-27T06:58:38.098628+0000","last_fullsized":"2024-05-27T06:58:38.098628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092627+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T06:58:38.092627+0000","last_peered":"2024-05-27T06:58:38.092627+0000","last_clean":"2024-05-27T06:58:38.092627+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T06:58:38.092627+0000","last_undegraded":"2024-05-27T06:58:38.092627+0000","last_fullsized":"2024-05-27T06:58:38.092627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102225+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T06:58:38.102225+0000","last_peered":"2024-05-27T06:58:38.102225+0000","last_clean":"2024-05-27T06:58:38.102225+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T06:58:38.102225+0000","last_undegraded":"2024-05-27T06:58:38.102225+0000","last_fullsized":"2024-05-27T06:58:38.102225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092171+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T06:58:38.092171+0000","last_peered":"2024-05-27T06:58:38.092171+0000","last_clean":"2024-05-27T06:58:38.092171+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T06:58:38.092171+0000","last_undegraded":"2024-05-27T06:58:38.092171+0000","last_fullsized":"2024-05-27T06:58:38.092171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.063473+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T06:58:38.063473+0000","last_peered":"2024-05-27T06:58:38.063473+0000","last_clean":"2024-05-27T06:58:38.063473+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T06:58:38.063473+0000","last_undegraded":"2024-05-27T06:58:38.063473+0000","last_fullsized":"2024-05-27T06:58:38.063473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.412340+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T06:56:40.412340+0000","last_peered":"2024-05-27T06:56:40.412340+0000","last_clean":"2024-05-27T06:56:40.412340+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T06:56:40.412340+0000","last_undegraded":"2024-05-27T06:56:40.412340+0000","last_fullsized":"2024-05-27T06:56:40.412340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":187,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:49.835574+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T06:58:49.835574+0000","last_peered":"2024-05-27T06:58:49.835574+0000","last_clean":"2024-05-27T06:58:49.835574+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T06:58:49.835574+0000","last_undegraded":"2024-05-27T06:58:49.835574+0000","last_fullsized":"2024-05-27T06:58:49.835574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.070739+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T06:58:38.070739+0000","last_peered":"2024-05-27T06:58:38.070739+0000","last_clean":"2024-05-27T06:58:38.070739+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T06:58:38.070739+0000","last_undegraded":"2024-05-27T06:58:38.070739+0000","last_fullsized":"2024-05-27T06:58:38.070739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408788+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T06:56:40.408788+0000","last_peered":"2024-05-27T06:56:40.408788+0000","last_clean":"2024-05-27T06:56:40.408788+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T06:56:40.408788+0000","last_undegraded":"2024-05-27T06:56:40.408788+0000","last_fullsized":"2024-05-27T06:56:40.408788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.411651+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T06:56:40.411651+0000","last_peered":"2024-05-27T06:56:40.411651+0000","last_clean":"2024-05-27T06:56:40.411651+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T06:56:40.411651+0000","last_undegraded":"2024-05-27T06:56:40.411651+0000","last_fullsized":"2024-05-27T06:56:40.411651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.102112+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T06:58:38.102112+0000","last_peered":"2024-05-27T06:58:38.102112+0000","last_clean":"2024-05-27T06:58:38.102112+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T06:58:38.102112+0000","last_undegraded":"2024-05-27T06:58:38.102112+0000","last_fullsized":"2024-05-27T06:58:38.102112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.341777+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T06:56:40.341777+0000","last_peered":"2024-05-27T06:56:40.341777+0000","last_clean":"2024-05-27T06:56:40.341777+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T06:56:40.341777+0000","last_undegraded":"2024-05-27T06:56:40.341777+0000","last_fullsized":"2024-05-27T06:56:40.341777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449941+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T06:56:44.449941+0000","last_peered":"2024-05-27T06:56:44.449941+0000","last_clean":"2024-05-27T06:56:44.449941+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T06:56:44.449941+0000","last_undegraded":"2024-05-27T06:56:44.449941+0000","last_fullsized":"2024-05-27T06:56:44.449941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.097861+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T06:58:38.097861+0000","last_peered":"2024-05-27T06:58:38.097861+0000","last_clean":"2024-05-27T06:58:38.097861+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T06:58:38.097861+0000","last_undegraded":"2024-05-27T06:58:38.097861+0000","last_fullsized":"2024-05-27T06:58:38.097861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408756+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T06:56:40.408756+0000","last_peered":"2024-05-27T06:56:40.408756+0000","last_clean":"2024-05-27T06:56:40.408756+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T06:56:40.408756+0000","last_undegraded":"2024-05-27T06:56:40.408756+0000","last_fullsized":"2024-05-27T06:56:40.408756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.457154+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T06:56:44.457154+0000","last_peered":"2024-05-27T06:56:44.457154+0000","last_clean":"2024-05-27T06:56:44.457154+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T06:56:44.457154+0000","last_undegraded":"2024-05-27T06:56:44.457154+0000","last_fullsized":"2024-05-27T06:56:44.457154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098417+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T06:58:38.098417+0000","last_peered":"2024-05-27T06:58:38.098417+0000","last_clean":"2024-05-27T06:58:38.098417+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T06:58:38.098417+0000","last_undegraded":"2024-05-27T06:58:38.098417+0000","last_fullsized":"2024-05-27T06:58:38.098417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349631+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T06:56:40.349631+0000","last_peered":"2024-05-27T06:56:40.349631+0000","last_clean":"2024-05-27T06:56:40.349631+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T06:56:40.349631+0000","last_undegraded":"2024-05-27T06:56:40.349631+0000","last_fullsized":"2024-05-27T06:56:40.349631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:49.857039+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T06:58:49.857039+0000","last_peered":"2024-05-27T06:58:49.857039+0000","last_clean":"2024-05-27T06:58:49.857039+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T06:58:49.857039+0000","last_undegraded":"2024-05-27T06:58:49.857039+0000","last_fullsized":"2024-05-27T06:58:49.857039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.079131+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T06:58:38.079131+0000","last_peered":"2024-05-27T06:58:38.079131+0000","last_clean":"2024-05-27T06:58:38.079131+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T06:58:38.079131+0000","last_undegraded":"2024-05-27T06:58:38.079131+0000","last_fullsized":"2024-05-27T06:58:38.079131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.349192+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T06:56:40.349192+0000","last_peered":"2024-05-27T06:56:40.349192+0000","last_clean":"2024-05-27T06:56:40.349192+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T06:56:40.349192+0000","last_undegraded":"2024-05-27T06:56:40.349192+0000","last_fullsized":"2024-05-27T06:56:40.349192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422145+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T06:56:41.422145+0000","last_peered":"2024-05-27T06:56:41.422145+0000","last_clean":"2024-05-27T06:56:41.422145+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T06:56:41.422145+0000","last_undegraded":"2024-05-27T06:56:41.422145+0000","last_fullsized":"2024-05-27T06:56:41.422145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077442+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T06:58:38.077442+0000","last_peered":"2024-05-27T06:58:38.077442+0000","last_clean":"2024-05-27T06:58:38.077442+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T06:58:38.077442+0000","last_undegraded":"2024-05-27T06:58:38.077442+0000","last_fullsized":"2024-05-27T06:58:38.077442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.408179+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T06:56:40.408179+0000","last_peered":"2024-05-27T06:56:40.408179+0000","last_clean":"2024-05-27T06:56:40.408179+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T06:56:40.408179+0000","last_undegraded":"2024-05-27T06:56:40.408179+0000","last_fullsized":"2024-05-27T06:56:40.408179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.435023+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T06:56:42.435023+0000","last_peered":"2024-05-27T06:56:42.435023+0000","last_clean":"2024-05-27T06:56:42.435023+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T06:56:42.435023+0000","last_undegraded":"2024-05-27T06:56:42.435023+0000","last_fullsized":"2024-05-27T06:56:42.435023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.069349+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T06:58:38.069349+0000","last_peered":"2024-05-27T06:58:38.069349+0000","last_clean":"2024-05-27T06:58:38.069349+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T06:58:38.069349+0000","last_undegraded":"2024-05-27T06:58:38.069349+0000","last_fullsized":"2024-05-27T06:58:38.069349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399556+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T06:56:40.399556+0000","last_peered":"2024-05-27T06:56:40.399556+0000","last_clean":"2024-05-27T06:56:40.399556+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T06:56:40.399556+0000","last_undegraded":"2024-05-27T06:56:40.399556+0000","last_fullsized":"2024-05-27T06:56:40.399556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.456993+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T06:56:44.456993+0000","last_peered":"2024-05-27T06:56:44.456993+0000","last_clean":"2024-05-27T06:56:44.456993+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T06:56:44.456993+0000","last_undegraded":"2024-05-27T06:56:44.456993+0000","last_fullsized":"2024-05-27T06:56:44.456993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.083741+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T06:58:38.083741+0000","last_peered":"2024-05-27T06:58:38.083741+0000","last_clean":"2024-05-27T06:58:38.083741+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T06:58:38.083741+0000","last_undegraded":"2024-05-27T06:58:38.083741+0000","last_fullsized":"2024-05-27T06:58:38.083741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402777+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T06:56:40.402777+0000","last_peered":"2024-05-27T06:56:40.402777+0000","last_clean":"2024-05-27T06:56:40.402777+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T06:56:40.402777+0000","last_undegraded":"2024-05-27T06:56:40.402777+0000","last_fullsized":"2024-05-27T06:56:40.402777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:51.862014+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T06:58:51.862014+0000","last_peered":"2024-05-27T06:58:51.862014+0000","last_clean":"2024-05-27T06:58:51.862014+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T06:58:51.862014+0000","last_undegraded":"2024-05-27T06:58:51.862014+0000","last_fullsized":"2024-05-27T06:58:51.862014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094922+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T06:58:38.094922+0000","last_peered":"2024-05-27T06:58:38.094922+0000","last_clean":"2024-05-27T06:58:38.094922+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T06:58:38.094922+0000","last_undegraded":"2024-05-27T06:58:38.094922+0000","last_fullsized":"2024-05-27T06:58:38.094922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.398619+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T06:56:40.398619+0000","last_peered":"2024-05-27T06:56:40.398619+0000","last_clean":"2024-05-27T06:56:40.398619+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T06:56:40.398619+0000","last_undegraded":"2024-05-27T06:56:40.398619+0000","last_fullsized":"2024-05-27T06:56:40.398619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434594+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T06:56:42.434594+0000","last_peered":"2024-05-27T06:56:42.434594+0000","last_clean":"2024-05-27T06:56:42.434594+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T06:56:42.434594+0000","last_undegraded":"2024-05-27T06:56:42.434594+0000","last_fullsized":"2024-05-27T06:56:42.434594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100225+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T06:58:38.100225+0000","last_peered":"2024-05-27T06:58:38.100225+0000","last_clean":"2024-05-27T06:58:38.100225+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T06:58:38.100225+0000","last_undegraded":"2024-05-27T06:58:38.100225+0000","last_fullsized":"2024-05-27T06:58:38.100225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342480+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T06:56:40.342480+0000","last_peered":"2024-05-27T06:56:40.342480+0000","last_clean":"2024-05-27T06:56:40.342480+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T06:56:40.342480+0000","last_undegraded":"2024-05-27T06:56:40.342480+0000","last_fullsized":"2024-05-27T06:56:40.342480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.456185+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T06:56:44.456185+0000","last_peered":"2024-05-27T06:56:44.456185+0000","last_clean":"2024-05-27T06:56:44.456185+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T06:56:44.456185+0000","last_undegraded":"2024-05-27T06:56:44.456185+0000","last_fullsized":"2024-05-27T06:56:44.456185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098075+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T06:58:38.098075+0000","last_peered":"2024-05-27T06:58:38.098075+0000","last_clean":"2024-05-27T06:58:38.098075+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T06:58:38.098075+0000","last_undegraded":"2024-05-27T06:58:38.098075+0000","last_fullsized":"2024-05-27T06:58:38.098075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.399820+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T06:56:40.399820+0000","last_peered":"2024-05-27T06:56:40.399820+0000","last_clean":"2024-05-27T06:56:40.399820+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T06:56:40.399820+0000","last_undegraded":"2024-05-27T06:56:40.399820+0000","last_fullsized":"2024-05-27T06:56:40.399820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'39","reported_seq":368,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:51.872685+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T06:58:51.872685+0000","last_peered":"2024-05-27T06:58:51.872685+0000","last_clean":"2024-05-27T06:58:51.872685+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T06:58:51.872685+0000","last_undegraded":"2024-05-27T06:58:51.872685+0000","last_fullsized":"2024-05-27T06:58:51.872685+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66730,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":199,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.074184+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T06:58:38.074184+0000","last_peered":"2024-05-27T06:58:38.074184+0000","last_clean":"2024-05-27T06:58:38.074184+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T06:58:38.074184+0000","last_undegraded":"2024-05-27T06:58:38.074184+0000","last_fullsized":"2024-05-27T06:58:38.074184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342280+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T06:56:40.342280+0000","last_peered":"2024-05-27T06:56:40.342280+0000","last_clean":"2024-05-27T06:56:40.342280+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T06:56:40.342280+0000","last_undegraded":"2024-05-27T06:56:40.342280+0000","last_fullsized":"2024-05-27T06:56:40.342280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451259+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T06:56:44.451259+0000","last_peered":"2024-05-27T06:56:44.451259+0000","last_clean":"2024-05-27T06:56:44.451259+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T06:56:44.451259+0000","last_undegraded":"2024-05-27T06:56:44.451259+0000","last_fullsized":"2024-05-27T06:56:44.451259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.083869+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T06:58:38.083869+0000","last_peered":"2024-05-27T06:58:38.083869+0000","last_clean":"2024-05-27T06:58:38.083869+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T06:58:38.083869+0000","last_undegraded":"2024-05-27T06:58:38.083869+0000","last_fullsized":"2024-05-27T06:58:38.083869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401558+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T06:56:40.401558+0000","last_peered":"2024-05-27T06:56:40.401558+0000","last_clean":"2024-05-27T06:56:40.401558+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T06:56:40.401558+0000","last_undegraded":"2024-05-27T06:56:40.401558+0000","last_fullsized":"2024-05-27T06:56:40.401558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449358+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T06:56:44.449358+0000","last_peered":"2024-05-27T06:56:44.449358+0000","last_clean":"2024-05-27T06:56:44.449358+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T06:56:44.449358+0000","last_undegraded":"2024-05-27T06:56:44.449358+0000","last_fullsized":"2024-05-27T06:56:44.449358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094375+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T06:58:38.094375+0000","last_peered":"2024-05-27T06:58:38.094375+0000","last_clean":"2024-05-27T06:58:38.094375+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T06:58:38.094375+0000","last_undegraded":"2024-05-27T06:58:38.094375+0000","last_fullsized":"2024-05-27T06:58:38.094375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402173+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T06:56:40.402173+0000","last_peered":"2024-05-27T06:56:40.402173+0000","last_clean":"2024-05-27T06:56:40.402173+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T06:56:40.402173+0000","last_undegraded":"2024-05-27T06:56:40.402173+0000","last_fullsized":"2024-05-27T06:56:40.402173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":184,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:50.871393+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T06:58:50.871393+0000","last_peered":"2024-05-27T06:58:50.871393+0000","last_clean":"2024-05-27T06:58:50.871393+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T06:58:50.871393+0000","last_undegraded":"2024-05-27T06:58:50.871393+0000","last_fullsized":"2024-05-27T06:58:50.871393+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.082881+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T06:58:38.082881+0000","last_peered":"2024-05-27T06:58:38.082881+0000","last_clean":"2024-05-27T06:58:38.082881+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T06:58:38.082881+0000","last_undegraded":"2024-05-27T06:58:38.082881+0000","last_fullsized":"2024-05-27T06:58:38.082881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335420+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T06:56:40.335420+0000","last_peered":"2024-05-27T06:56:40.335420+0000","last_clean":"2024-05-27T06:56:40.335420+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T06:56:40.335420+0000","last_undegraded":"2024-05-27T06:56:40.335420+0000","last_fullsized":"2024-05-27T06:56:40.335420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449699+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T06:56:44.449699+0000","last_peered":"2024-05-27T06:56:44.449699+0000","last_clean":"2024-05-27T06:56:44.449699+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T06:56:44.449699+0000","last_undegraded":"2024-05-27T06:56:44.449699+0000","last_fullsized":"2024-05-27T06:56:44.449699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096423+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T06:58:38.096423+0000","last_peered":"2024-05-27T06:58:38.096423+0000","last_clean":"2024-05-27T06:58:38.096423+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T06:58:38.096423+0000","last_undegraded":"2024-05-27T06:58:38.096423+0000","last_fullsized":"2024-05-27T06:58:38.096423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.340204+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T06:56:40.340204+0000","last_peered":"2024-05-27T06:56:40.340204+0000","last_clean":"2024-05-27T06:56:40.340204+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T06:56:40.340204+0000","last_undegraded":"2024-05-27T06:56:40.340204+0000","last_fullsized":"2024-05-27T06:56:40.340204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451292+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T06:56:44.451292+0000","last_peered":"2024-05-27T06:56:44.451292+0000","last_clean":"2024-05-27T06:56:44.451292+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T06:56:44.451292+0000","last_undegraded":"2024-05-27T06:56:44.451292+0000","last_fullsized":"2024-05-27T06:56:44.451292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075829+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T06:58:38.075829+0000","last_peered":"2024-05-27T06:58:38.075829+0000","last_clean":"2024-05-27T06:58:38.075829+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T06:58:38.075829+0000","last_undegraded":"2024-05-27T06:58:38.075829+0000","last_fullsized":"2024-05-27T06:58:38.075829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401521+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T06:56:40.401521+0000","last_peered":"2024-05-27T06:56:40.401521+0000","last_clean":"2024-05-27T06:56:40.401521+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T06:56:40.401521+0000","last_undegraded":"2024-05-27T06:56:40.401521+0000","last_fullsized":"2024-05-27T06:56:40.401521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:41.422923+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T06:56:41.422923+0000","last_peered":"2024-05-27T06:56:41.422923+0000","last_clean":"2024-05-27T06:56:41.422923+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T06:56:41.422923+0000","last_undegraded":"2024-05-27T06:56:41.422923+0000","last_fullsized":"2024-05-27T06:56:41.422923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087790+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T06:58:38.087790+0000","last_peered":"2024-05-27T06:58:38.087790+0000","last_clean":"2024-05-27T06:58:38.087790+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T06:58:38.087790+0000","last_undegraded":"2024-05-27T06:58:38.087790+0000","last_fullsized":"2024-05-27T06:58:38.087790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335922+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T06:56:40.335922+0000","last_peered":"2024-05-27T06:56:40.335922+0000","last_clean":"2024-05-27T06:56:40.335922+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T06:56:40.335922+0000","last_undegraded":"2024-05-27T06:56:40.335922+0000","last_fullsized":"2024-05-27T06:56:40.335922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":185,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:52.846957+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T06:58:52.846957+0000","last_peered":"2024-05-27T06:58:52.846957+0000","last_clean":"2024-05-27T06:58:52.846957+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T06:58:52.846957+0000","last_undegraded":"2024-05-27T06:58:52.846957+0000","last_fullsized":"2024-05-27T06:58:52.846957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.090877+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T06:58:38.090877+0000","last_peered":"2024-05-27T06:58:38.090877+0000","last_clean":"2024-05-27T06:58:38.090877+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T06:58:38.090877+0000","last_undegraded":"2024-05-27T06:58:38.090877+0000","last_fullsized":"2024-05-27T06:58:38.090877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.335077+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T06:56:40.335077+0000","last_peered":"2024-05-27T06:56:40.335077+0000","last_clean":"2024-05-27T06:56:40.335077+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T06:56:40.335077+0000","last_undegraded":"2024-05-27T06:56:40.335077+0000","last_fullsized":"2024-05-27T06:56:40.335077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076770+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T06:58:38.076770+0000","last_peered":"2024-05-27T06:58:38.076770+0000","last_clean":"2024-05-27T06:58:38.076770+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T06:58:38.076770+0000","last_undegraded":"2024-05-27T06:58:38.076770+0000","last_fullsized":"2024-05-27T06:58:38.076770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":180,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:50.869921+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T06:58:50.869921+0000","last_peered":"2024-05-27T06:58:50.869921+0000","last_clean":"2024-05-27T06:58:50.869921+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T06:58:50.869921+0000","last_undegraded":"2024-05-27T06:58:50.869921+0000","last_fullsized":"2024-05-27T06:58:50.869921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:57.726620+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T06:56:57.726620+0000","last_peered":"2024-05-27T06:56:57.726620+0000","last_clean":"2024-05-27T06:56:57.726620+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T06:56:57.726620+0000","last_undegraded":"2024-05-27T06:56:57.726620+0000","last_fullsized":"2024-05-27T06:56:57.726620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.092774+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T06:58:38.092774+0000","last_peered":"2024-05-27T06:58:38.092774+0000","last_clean":"2024-05-27T06:58:38.092774+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T06:58:38.092774+0000","last_undegraded":"2024-05-27T06:58:38.092774+0000","last_fullsized":"2024-05-27T06:58:38.092774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450805+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T06:56:44.450805+0000","last_peered":"2024-05-27T06:56:44.450805+0000","last_clean":"2024-05-27T06:56:44.450805+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T06:56:44.450805+0000","last_undegraded":"2024-05-27T06:56:44.450805+0000","last_fullsized":"2024-05-27T06:56:44.450805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.337815+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T06:56:40.337815+0000","last_peered":"2024-05-27T06:56:40.337815+0000","last_clean":"2024-05-27T06:56:40.337815+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T06:56:40.337815+0000","last_undegraded":"2024-05-27T06:56:40.337815+0000","last_fullsized":"2024-05-27T06:56:40.337815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.342212+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T06:56:40.342212+0000","last_peered":"2024-05-27T06:56:40.342212+0000","last_clean":"2024-05-27T06:56:40.342212+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T06:56:40.342212+0000","last_undegraded":"2024-05-27T06:56:40.342212+0000","last_fullsized":"2024-05-27T06:56:40.342212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077957+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T06:58:38.077957+0000","last_peered":"2024-05-27T06:58:38.077957+0000","last_clean":"2024-05-27T06:58:38.077957+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T06:58:38.077957+0000","last_undegraded":"2024-05-27T06:58:38.077957+0000","last_fullsized":"2024-05-27T06:58:38.077957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432537+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T06:56:42.432537+0000","last_peered":"2024-05-27T06:56:42.432537+0000","last_clean":"2024-05-27T06:56:42.432537+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T06:56:42.432537+0000","last_undegraded":"2024-05-27T06:56:42.432537+0000","last_fullsized":"2024-05-27T06:56:42.432537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402603+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T06:56:40.402603+0000","last_peered":"2024-05-27T06:56:40.402603+0000","last_clean":"2024-05-27T06:56:40.402603+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T06:56:40.402603+0000","last_undegraded":"2024-05-27T06:56:40.402603+0000","last_fullsized":"2024-05-27T06:56:40.402603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.450385+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T06:56:44.450385+0000","last_peered":"2024-05-27T06:56:44.450385+0000","last_clean":"2024-05-27T06:56:44.450385+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T06:56:44.450385+0000","last_undegraded":"2024-05-27T06:56:44.450385+0000","last_fullsized":"2024-05-27T06:56:44.450385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.078740+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T06:58:38.078740+0000","last_peered":"2024-05-27T06:58:38.078740+0000","last_clean":"2024-05-27T06:58:38.078740+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T06:58:38.078740+0000","last_undegraded":"2024-05-27T06:58:38.078740+0000","last_fullsized":"2024-05-27T06:58:38.078740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.404013+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T06:56:40.404013+0000","last_peered":"2024-05-27T06:56:40.404013+0000","last_clean":"2024-05-27T06:56:40.404013+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T06:56:40.404013+0000","last_undegraded":"2024-05-27T06:56:40.404013+0000","last_fullsized":"2024-05-27T06:56:40.404013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'17","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:17.444934+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T06:57:17.444934+0000","last_peered":"2024-05-27T06:57:17.444934+0000","last_clean":"2024-05-27T06:57:17.444934+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T06:57:17.444934+0000","last_undegraded":"2024-05-27T06:57:17.444934+0000","last_fullsized":"2024-05-27T06:57:17.444934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087132+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T06:58:38.087132+0000","last_peered":"2024-05-27T06:58:38.087132+0000","last_clean":"2024-05-27T06:58:38.087132+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T06:58:38.087132+0000","last_undegraded":"2024-05-27T06:58:38.087132+0000","last_fullsized":"2024-05-27T06:58:38.087132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.348360+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T06:56:40.348360+0000","last_peered":"2024-05-27T06:56:40.348360+0000","last_clean":"2024-05-27T06:56:40.348360+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T06:56:40.348360+0000","last_undegraded":"2024-05-27T06:56:40.348360+0000","last_fullsized":"2024-05-27T06:56:40.348360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.449227+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T06:56:44.449227+0000","last_peered":"2024-05-27T06:56:44.449227+0000","last_clean":"2024-05-27T06:56:44.449227+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T06:56:44.449227+0000","last_undegraded":"2024-05-27T06:56:44.449227+0000","last_fullsized":"2024-05-27T06:56:44.449227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094673+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T06:58:38.094673+0000","last_peered":"2024-05-27T06:58:38.094673+0000","last_clean":"2024-05-27T06:58:38.094673+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T06:58:38.094673+0000","last_undegraded":"2024-05-27T06:58:38.094673+0000","last_fullsized":"2024-05-27T06:58:38.094673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344207+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T06:56:40.344207+0000","last_peered":"2024-05-27T06:56:40.344207+0000","last_clean":"2024-05-27T06:56:40.344207+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T06:56:40.344207+0000","last_undegraded":"2024-05-27T06:56:40.344207+0000","last_fullsized":"2024-05-27T06:56:40.344207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.394827+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T06:56:40.394827+0000","last_peered":"2024-05-27T06:56:40.394827+0000","last_clean":"2024-05-27T06:56:40.394827+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T06:56:40.394827+0000","last_undegraded":"2024-05-27T06:56:40.394827+0000","last_fullsized":"2024-05-27T06:56:40.394827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.025373+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T06:58:38.025373+0000","last_peered":"2024-05-27T06:58:38.025373+0000","last_clean":"2024-05-27T06:58:38.025373+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T06:58:38.025373+0000","last_undegraded":"2024-05-27T06:58:38.025373+0000","last_fullsized":"2024-05-27T06:58:38.025373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.402580+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T06:56:40.402580+0000","last_peered":"2024-05-27T06:56:40.402580+0000","last_clean":"2024-05-27T06:56:40.402580+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T06:56:40.402580+0000","last_undegraded":"2024-05-27T06:56:40.402580+0000","last_fullsized":"2024-05-27T06:56:40.402580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.434757+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T06:56:42.434757+0000","last_peered":"2024-05-27T06:56:42.434757+0000","last_clean":"2024-05-27T06:56:42.434757+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T06:56:42.434757+0000","last_undegraded":"2024-05-27T06:56:42.434757+0000","last_fullsized":"2024-05-27T06:56:42.434757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.100321+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T06:58:38.100321+0000","last_peered":"2024-05-27T06:58:38.100321+0000","last_clean":"2024-05-27T06:58:38.100321+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T06:58:38.100321+0000","last_undegraded":"2024-05-27T06:58:38.100321+0000","last_fullsized":"2024-05-27T06:58:38.100321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:57:06.295632+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T06:57:06.295632+0000","last_peered":"2024-05-27T06:57:06.295632+0000","last_clean":"2024-05-27T06:57:06.295632+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T06:57:06.295632+0000","last_undegraded":"2024-05-27T06:57:06.295632+0000","last_fullsized":"2024-05-27T06:57:06.295632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.407953+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T06:56:42.407953+0000","last_peered":"2024-05-27T06:56:42.407953+0000","last_clean":"2024-05-27T06:56:42.407953+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T06:56:42.407953+0000","last_undegraded":"2024-05-27T06:56:42.407953+0000","last_fullsized":"2024-05-27T06:56:42.407953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.090494+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T06:58:38.090494+0000","last_peered":"2024-05-27T06:58:38.090494+0000","last_clean":"2024-05-27T06:58:38.090494+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T06:58:38.090494+0000","last_undegraded":"2024-05-27T06:58:38.090494+0000","last_fullsized":"2024-05-27T06:58:38.090494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.344102+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T06:56:40.344102+0000","last_peered":"2024-05-27T06:56:40.344102+0000","last_clean":"2024-05-27T06:56:40.344102+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T06:56:40.344102+0000","last_undegraded":"2024-05-27T06:56:40.344102+0000","last_fullsized":"2024-05-27T06:56:40.344102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451041+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T06:56:44.451041+0000","last_peered":"2024-05-27T06:56:44.451041+0000","last_clean":"2024-05-27T06:56:44.451041+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T06:56:44.451041+0000","last_undegraded":"2024-05-27T06:56:44.451041+0000","last_fullsized":"2024-05-27T06:56:44.451041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093437+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T06:58:38.093437+0000","last_peered":"2024-05-27T06:58:38.093437+0000","last_clean":"2024-05-27T06:58:38.093437+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T06:58:38.093437+0000","last_undegraded":"2024-05-27T06:58:38.093437+0000","last_fullsized":"2024-05-27T06:58:38.093437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.396247+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T06:56:40.396247+0000","last_peered":"2024-05-27T06:56:40.396247+0000","last_clean":"2024-05-27T06:56:40.396247+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T06:56:40.396247+0000","last_undegraded":"2024-05-27T06:56:40.396247+0000","last_fullsized":"2024-05-27T06:56:40.396247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":186,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:52.882159+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T06:58:52.882159+0000","last_peered":"2024-05-27T06:58:52.882159+0000","last_clean":"2024-05-27T06:58:52.882159+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T06:58:52.882159+0000","last_undegraded":"2024-05-27T06:58:52.882159+0000","last_fullsized":"2024-05-27T06:58:52.882159+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.093626+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T06:58:38.093626+0000","last_peered":"2024-05-27T06:58:38.093626+0000","last_clean":"2024-05-27T06:58:38.093626+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T06:58:38.093626+0000","last_undegraded":"2024-05-27T06:58:38.093626+0000","last_fullsized":"2024-05-27T06:58:38.093626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.401123+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T06:56:40.401123+0000","last_peered":"2024-05-27T06:56:40.401123+0000","last_clean":"2024-05-27T06:56:40.401123+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T06:56:40.401123+0000","last_undegraded":"2024-05-27T06:56:40.401123+0000","last_fullsized":"2024-05-27T06:56:40.401123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:44.451295+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T06:56:44.451295+0000","last_peered":"2024-05-27T06:56:44.451295+0000","last_clean":"2024-05-27T06:56:44.451295+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T06:56:44.451295+0000","last_undegraded":"2024-05-27T06:56:44.451295+0000","last_fullsized":"2024-05-27T06:56:44.451295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.073707+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T06:58:38.073707+0000","last_peered":"2024-05-27T06:58:38.073707+0000","last_clean":"2024-05-27T06:58:38.073707+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T06:58:38.073707+0000","last_undegraded":"2024-05-27T06:58:38.073707+0000","last_fullsized":"2024-05-27T06:58:38.073707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:40.339817+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T06:56:40.339817+0000","last_peered":"2024-05-27T06:56:40.339817+0000","last_clean":"2024-05-27T06:56:40.339817+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T06:56:40.339817+0000","last_undegraded":"2024-05-27T06:56:40.339817+0000","last_fullsized":"2024-05-27T06:56:40.339817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:56:42.432981+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T06:56:42.432981+0000","last_peered":"2024-05-27T06:56:42.432981+0000","last_clean":"2024-05-27T06:56:42.432981+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T06:56:42.432981+0000","last_undegraded":"2024-05-27T06:56:42.432981+0000","last_fullsized":"2024-05-27T06:56:42.432981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.089291+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T06:58:38.089291+0000","last_peered":"2024-05-27T06:58:38.089291+0000","last_clean":"2024-05-27T06:58:38.089291+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T06:58:38.089291+0000","last_undegraded":"2024-05-27T06:58:38.089291+0000","last_fullsized":"2024-05-27T06:58:38.089291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076863+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T06:58:38.076863+0000","last_peered":"2024-05-27T06:58:38.076863+0000","last_clean":"2024-05-27T06:58:38.076863+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T06:58:38.076863+0000","last_undegraded":"2024-05-27T06:58:38.076863+0000","last_fullsized":"2024-05-27T06:58:38.076863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.075225+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T06:58:38.075225+0000","last_peered":"2024-05-27T06:58:38.075225+0000","last_clean":"2024-05-27T06:58:38.075225+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T06:58:38.075225+0000","last_undegraded":"2024-05-27T06:58:38.075225+0000","last_fullsized":"2024-05-27T06:58:38.075225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.101321+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T06:58:38.101321+0000","last_peered":"2024-05-27T06:58:38.101321+0000","last_clean":"2024-05-27T06:58:38.101321+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T06:58:38.101321+0000","last_undegraded":"2024-05-27T06:58:38.101321+0000","last_fullsized":"2024-05-27T06:58:38.101321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.098334+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T06:58:38.098334+0000","last_peered":"2024-05-27T06:58:38.098334+0000","last_clean":"2024-05-27T06:58:38.098334+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T06:58:38.098334+0000","last_undegraded":"2024-05-27T06:58:38.098334+0000","last_fullsized":"2024-05-27T06:58:38.098334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096554+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T06:58:38.096554+0000","last_peered":"2024-05-27T06:58:38.096554+0000","last_clean":"2024-05-27T06:58:38.096554+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T06:58:38.096554+0000","last_undegraded":"2024-05-27T06:58:38.096554+0000","last_fullsized":"2024-05-27T06:58:38.096554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.076044+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T06:58:38.076044+0000","last_peered":"2024-05-27T06:58:38.076044+0000","last_clean":"2024-05-27T06:58:38.076044+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T06:58:38.076044+0000","last_undegraded":"2024-05-27T06:58:38.076044+0000","last_fullsized":"2024-05-27T06:58:38.076044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.087261+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T06:58:38.087261+0000","last_peered":"2024-05-27T06:58:38.087261+0000","last_clean":"2024-05-27T06:58:38.087261+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T06:58:38.087261+0000","last_undegraded":"2024-05-27T06:58:38.087261+0000","last_fullsized":"2024-05-27T06:58:38.087261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.066236+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T06:58:38.066236+0000","last_peered":"2024-05-27T06:58:38.066236+0000","last_clean":"2024-05-27T06:58:38.066236+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T06:58:38.066236+0000","last_undegraded":"2024-05-27T06:58:38.066236+0000","last_fullsized":"2024-05-27T06:58:38.066236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.080408+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T06:58:38.080408+0000","last_peered":"2024-05-27T06:58:38.080408+0000","last_clean":"2024-05-27T06:58:38.080408+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T06:58:38.080408+0000","last_undegraded":"2024-05-27T06:58:38.080408+0000","last_fullsized":"2024-05-27T06:58:38.080408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.067486+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T06:58:38.067486+0000","last_peered":"2024-05-27T06:58:38.067486+0000","last_clean":"2024-05-27T06:58:38.067486+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T06:58:38.067486+0000","last_undegraded":"2024-05-27T06:58:38.067486+0000","last_fullsized":"2024-05-27T06:58:38.067486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.094527+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T06:58:38.094527+0000","last_peered":"2024-05-27T06:58:38.094527+0000","last_clean":"2024-05-27T06:58:38.094527+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T06:58:38.094527+0000","last_undegraded":"2024-05-27T06:58:38.094527+0000","last_fullsized":"2024-05-27T06:58:38.094527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.096850+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T06:58:38.096850+0000","last_peered":"2024-05-27T06:58:38.096850+0000","last_clean":"2024-05-27T06:58:38.096850+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T06:58:38.096850+0000","last_undegraded":"2024-05-27T06:58:38.096850+0000","last_fullsized":"2024-05-27T06:58:38.096850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-27T06:58:38.077760+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T06:58:38.077760+0000","last_peered":"2024-05-27T06:58:38.077760+0000","last_clean":"2024-05-27T06:58:38.077760+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T06:58:38.077760+0000","last_undegraded":"2024-05-27T06:58:38.077760+0000","last_fullsized":"2024-05-27T06:58:38.077760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74748,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1330,"num_read_kb":746,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":224244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874313,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1175147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037841,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234072,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":521377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430305,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1662976,"data_stored":1118547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626535,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":518491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182525,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986294,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":587839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822767,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T06:58:57.480 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T06:58:57.480 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T06:58:57.480 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T06:58:57.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T06:58:57.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:57 smithi164 ceph-mon[35327]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T06:58:57.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:57 smithi164 ceph-mon[35327]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:57.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:57 smithi016 ceph-mon[31415]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T06:58:57.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:57 smithi016 ceph-mon[31415]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:57.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:57 smithi069 ceph-mon[35276]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T06:58:57.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:57 smithi069 ceph-mon[35276]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T06:58:57.992 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:58:59.672 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:58:59 smithi069 ceph-mon[35276]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:58:59.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:58:59 smithi164 ceph-mon[35327]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:58:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:58:59 smithi016 ceph-mon[31415]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 -- 172.21.15.16:0/3720109144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c10a980 msgr2=0x7f2b1c10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 --2- 172.21.15.16:0/3720109144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c10a980 0x7f2b1c10ad90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009990 tx=0x7f2b08009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 -- 172.21.15.16:0/3720109144 shutdown_connections 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 --2- 172.21.15.16:0/3720109144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10b2d0 0x7f2b1c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 --2- 172.21.15.16:0/3720109144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c110960 0x7f2b1c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 --2- 172.21.15.16:0/3720109144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c10a980 0x7f2b1c10ad90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 -- 172.21.15.16:0/3720109144 >> 172.21.15.16:0/3720109144 conn(0x7f2b1c077c70 msgr2=0x7f2b1c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 -- 172.21.15.16:0/3720109144 shutdown_connections 2024-05-27T06:59:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.444+0000 7f2b24800700 1 -- 172.21.15.16:0/3720109144 wait complete. 2024-05-27T06:59:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.446+0000 7f2b24800700 1 Processor -- start 2024-05-27T06:59:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.446+0000 7f2b24800700 1 -- start start 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10a980 0x7f2b1c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c10b2d0 0x7f2b1c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c11a930 con 0x7f2b1c110960 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c11a7b0 con 0x7f2b1c10b2d0 2024-05-27T06:59:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b24800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c11aab0 con 0x7f2b1c10a980 2024-05-27T06:59:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b22800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b22800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41534/0 (socket says 172.21.15.16:41534) 2024-05-27T06:59:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b22800700 1 -- 172.21.15.16:0/3906653576 learned_addr learned my addr 172.21.15.16:0/3906653576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b21e00700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10a980 0x7f2b1c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.447+0000 7f2b21400700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c10b2d0 0x7f2b1c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 -- 172.21.15.16:0/3906653576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10a980 msgr2=0x7f2b1c1a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10a980 0x7f2b1c1a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 -- 172.21.15.16:0/3906653576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c10b2d0 msgr2=0x7f2b1c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c10b2d0 0x7f2b1c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 -- 172.21.15.16:0/3906653576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c1a6230 con 0x7f2b1c110960 2024-05-27T06:59:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.448+0000 7f2b22800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007ee0 tx=0x7f2b1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.449+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18004bb0 con 0x7f2b1c110960 2024-05-27T06:59:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.449+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b18004d50 con 0x7f2b1c110960 2024-05-27T06:59:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.449+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c1a6520 con 0x7f2b1c110960 2024-05-27T06:59:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.449+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180058b0 con 0x7f2b1c110960 2024-05-27T06:59:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.450+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1ade40 con 0x7f2b1c110960 2024-05-27T06:59:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.451+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c10c070 con 0x7f2b1c110960 2024-05-27T06:59:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.453+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2b1801a020 con 0x7f2b1c110960 2024-05-27T06:59:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.455+0000 7f2b16a00700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 0x7f2b040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.455+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b180e16e0 con 0x7f2b1c110960 2024-05-27T06:59:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.455+0000 7f2b21e00700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 0x7f2b040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.456+0000 7f2b21e00700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 0x7f2b040b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005950 tx=0x7f2b1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.460+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b18010070 con 0x7f2b1c110960 2024-05-27T06:59:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.746+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2b1c1a26c0 con 0x7f2b1c110960 2024-05-27T06:59:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.747+0000 7f2b16a00700 1 -- 172.21.15.16:0/3906653576 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2b180a8ff0 con 0x7f2b1c110960 2024-05-27T06:59:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:59:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T06:59:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 msgr2=0x7f2b040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 0x7f2b040b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005950 tx=0x7f2b1000b410 comp rx=0 tx=0).stop 2024-05-27T06:59:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 msgr2=0x7f2b1c1a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007ee0 tx=0x7f2b1800d3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 shutdown_connections 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f2b040b2880 0x7f2b040b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c110960 0x7f2b1c1a5b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b1c10a980 0x7f2b1c1a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 --2- 172.21.15.16:0/3906653576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b1c10b2d0 0x7f2b1c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 >> 172.21.15.16:0/3906653576 conn(0x7f2b1c077c70 msgr2=0x7f2b1c07b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.751+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 shutdown_connections 2024-05-27T06:59:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:00.752+0000 7f2b24800700 1 -- 172.21.15.16:0/3906653576 wait complete. 2024-05-27T06:59:01.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:01 smithi016 ceph-mon[31415]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-27T06:59:01.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:01 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3906653576' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T06:59:01.661 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T06:59:01.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:59:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:01 smithi164 ceph-mon[35327]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-27T06:59:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:01 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3906653576' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T06:59:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:01 smithi069 ceph-mon[35276]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-27T06:59:01.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:01 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3906653576' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T06:59:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:03.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:03 smithi016 ceph-mon[31415]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T06:59:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:03 smithi164 ceph-mon[35327]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T06:59:03.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:03 smithi069 ceph-mon[35276]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T06:59:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 -- 172.21.15.16:0/2175410878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d810a980 msgr2=0x7fa9d810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 --2- 172.21.15.16:0/2175410878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d810a980 0x7fa9d810ad90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009990 tx=0x7fa9c0009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 -- 172.21.15.16:0/2175410878 shutdown_connections 2024-05-27T06:59:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 --2- 172.21.15.16:0/2175410878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 0x7fa9d8117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 --2- 172.21.15.16:0/2175410878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8110960 0x7fa9d8110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 --2- 172.21.15.16:0/2175410878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d810a980 0x7fa9d810ad90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 -- 172.21.15.16:0/2175410878 >> 172.21.15.16:0/2175410878 conn(0x7fa9d8077c70 msgr2=0x7fa9d80780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 -- 172.21.15.16:0/2175410878 shutdown_connections 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.549+0000 7fa9de200700 1 -- 172.21.15.16:0/2175410878 wait complete. 2024-05-27T06:59:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 Processor -- start 2024-05-27T06:59:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 -- start start 2024-05-27T06:59:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d810a980 0x7fa9d81a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 0x7fa9d81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d811a910 con 0x7fa9d8110960 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d811a790 con 0x7fa9d810b2d0 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.551+0000 7fa9de200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d811aa90 con 0x7fa9d810a980 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.552+0000 7fa9d7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.552+0000 7fa9d7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43156/0 (socket says 172.21.15.16:43156) 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.552+0000 7fa9d7e00700 1 -- 172.21.15.16:0/1654075804 learned_addr learned my addr 172.21.15.16:0/1654075804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.552+0000 7fa9cfe00700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 0x7fa9d81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.552+0000 7fa9d7400700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d810a980 0x7fa9d81a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 -- 172.21.15.16:0/1654075804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d810a980 msgr2=0x7fa9d81a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d810a980 0x7fa9d81a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 -- 172.21.15.16:0/1654075804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 msgr2=0x7fa9d81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 0x7fa9d81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 -- 172.21.15.16:0/1654075804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a6100 con 0x7fa9d8110960 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d7e00700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0007d70 tx=0x7fa9d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0016020 con 0x7fa9d8110960 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9d000ac90 con 0x7fa9d8110960 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a63f0 con 0x7fa9d8110960 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.553+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00115e0 con 0x7fa9d8110960 2024-05-27T06:59:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.554+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a6900 con 0x7fa9d8110960 2024-05-27T06:59:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.555+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d80512b0 con 0x7fa9d8110960 2024-05-27T06:59:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.557+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa9d0019020 con 0x7fa9d8110960 2024-05-27T06:59:04.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.560+0000 7fa9d4c00700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 0x7fa9bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.560+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa9d00e26f0 con 0x7fa9d8110960 2024-05-27T06:59:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.560+0000 7fa9d7400700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 0x7fa9bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.561+0000 7fa9d7400700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 0x7fa9bc0b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d807d0e0 tx=0x7fa9c800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:04.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.563+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9d0020030 con 0x7fa9d8110960 2024-05-27T06:59:04.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.856+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9d802e5e0 con 0x7fa9d8110960 2024-05-27T06:59:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.858+0000 7fa9d4c00700 1 -- 172.21.15.16:0/1654075804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7fa9d00aa000 con 0x7fa9d8110960 2024-05-27T06:59:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:59:04.851 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T06:59:04.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 msgr2=0x7fa9bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:04.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 0x7fa9bc0b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9d807d0e0 tx=0x7fa9c800a380 comp rx=0 tx=0).stop 2024-05-27T06:59:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 msgr2=0x7fa9d81a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0007d70 tx=0x7fa9d000a340 comp rx=0 tx=0).stop 2024-05-27T06:59:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 shutdown_connections 2024-05-27T06:59:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fa9bc0b2900 0x7fa9bc0b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8110960 0x7fa9d81a5b10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d810a980 0x7fa9d81a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 --2- 172.21.15.16:0/1654075804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d810b2d0 0x7fa9d81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.861+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 >> 172.21.15.16:0/1654075804 conn(0x7fa9d8077c70 msgr2=0x7fa9d807b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.862+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 shutdown_connections 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:04.862+0000 7fa9de200700 1 -- 172.21.15.16:0/1654075804 wait complete. 2024-05-27T06:59:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T06:59:05.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:05 smithi016 ceph-mon[31415]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-27T06:59:05.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:05 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1654075804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:05.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:59:05.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:05 smithi164 ceph-mon[35327]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-27T06:59:05.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:05 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1654075804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:05.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:05 smithi069 ceph-mon[35276]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-27T06:59:05.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:05 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1654075804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:07.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:07 smithi016 ceph-mon[31415]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:07 smithi164 ceph-mon[35327]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:07.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:07 smithi069 ceph-mon[35276]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 -- 172.21.15.16:0/1822964717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 msgr2=0x7fe7d0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 --2- 172.21.15.16:0/1822964717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d0106a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40099c0 tx=0x7fe7c4009cd0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 -- 172.21.15.16:0/1822964717 shutdown_connections 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 --2- 172.21.15.16:0/1822964717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010b890 0x7fe7d01131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 --2- 172.21.15.16:0/1822964717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d0106a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 --2- 172.21.15.16:0/1822964717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010af40 0x7fe7d010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 -- 172.21.15.16:0/1822964717 >> 172.21.15.16:0/1822964717 conn(0x7fe7d0102020 msgr2=0x7fe7d0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 -- 172.21.15.16:0/1822964717 shutdown_connections 2024-05-27T06:59:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.949+0000 7fe7d6200700 1 -- 172.21.15.16:0/1822964717 wait complete. 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 Processor -- start 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 -- start start 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010af40 0x7fe7d019d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b890 0x7fe7d01a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0116370 con 0x7fe7d0106610 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01161f0 con 0x7fe7d010af40 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.951+0000 7fe7d6200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01164f0 con 0x7fe7d010b890 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43168/0 (socket says 172.21.15.16:43168) 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 -- 172.21.15.16:0/154933651 learned_addr learned my addr 172.21.15.16:0/154933651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7cfe00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010af40 0x7fe7d019d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d5800700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b890 0x7fe7d01a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 -- 172.21.15.16:0/154933651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b890 msgr2=0x7fe7d01a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b890 0x7fe7d01a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 -- 172.21.15.16:0/154933651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010af40 msgr2=0x7fe7d019d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010af40 0x7fe7d019d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.952+0000 7fe7d4e00700 1 -- 172.21.15.16:0/154933651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01a1b40 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7d4e00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098e0 tx=0x7fe7c400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4027070 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7c40056d0 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a1dd0 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40173f0 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.953+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a2300 con 0x7fe7d0106610 2024-05-27T06:59:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.954+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d00512b0 con 0x7fe7d0106610 2024-05-27T06:59:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.957+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe7c402a070 con 0x7fe7d0106610 2024-05-27T06:59:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.958+0000 7fe7cd600700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 0x7fe7b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.959+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe7c40e3630 con 0x7fe7d0106610 2024-05-27T06:59:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.959+0000 7fe7cfe00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 0x7fe7b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.960+0000 7fe7cfe00700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 0x7fe7b80b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005ea0 tx=0x7fe7bc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:08.961+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7c4013070 con 0x7fe7d0106610 2024-05-27T06:59:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.256+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d019e080 con 0x7fe7d0106610 2024-05-27T06:59:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.257+0000 7fe7cd600700 1 -- 172.21.15.16:0/154933651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7fe7c40aafd0 con 0x7fe7d0106610 2024-05-27T06:59:09.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:59:09.250 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T06:59:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 msgr2=0x7fe7b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 0x7fe7b80b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005ea0 tx=0x7fe7bc005e30 comp rx=0 tx=0).stop 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 msgr2=0x7fe7d019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098e0 tx=0x7fe7c400f740 comp rx=0 tx=0).stop 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 shutdown_connections 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fe7b80b2900 0x7fe7b80b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d010b890 0x7fe7d01a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0106610 0x7fe7d019cd20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:09.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 --2- 172.21.15.16:0/154933651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7d010af40 0x7fe7d019d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:09.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.261+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 >> 172.21.15.16:0/154933651 conn(0x7fe7d0102020 msgr2=0x7fe7d010a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:09.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.262+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 shutdown_connections 2024-05-27T06:59:09.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:09.262+0000 7fe7d6200700 1 -- 172.21.15.16:0/154933651 wait complete. 2024-05-27T06:59:09.255 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T06:59:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:09 smithi016 ceph-mon[31415]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:09.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:09 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/154933651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:09 smithi164 ceph-mon[35327]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:09 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/154933651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:09 smithi069 ceph-mon[35276]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:09.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:09 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/154933651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:10.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T06:59:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:11.716 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:11 smithi069 ceph-mon[35276]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:11.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:11 smithi016 ceph-mon[31415]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:11 smithi164 ceph-mon[35327]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.172+0000 7f1e71800700 1 -- 172.21.15.16:0/956631611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c106570 msgr2=0x7f1e6c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.172+0000 7f1e71800700 1 --2- 172.21.15.16:0/956631611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c106570 0x7f1e6c106980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009990 tx=0x7f1e58009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 -- 172.21.15.16:0/956631611 shutdown_connections 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 --2- 172.21.15.16:0/956631611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 --2- 172.21.15.16:0/956631611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 --2- 172.21.15.16:0/956631611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c106570 0x7f1e6c106980 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 -- 172.21.15.16:0/956631611 >> 172.21.15.16:0/956631611 conn(0x7f1e6c102070 msgr2=0x7f1e6c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 -- 172.21.15.16:0/956631611 shutdown_connections 2024-05-27T06:59:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.173+0000 7f1e71800700 1 -- 172.21.15.16:0/956631611 wait complete. 2024-05-27T06:59:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.174+0000 7f1e71800700 1 Processor -- start 2024-05-27T06:59:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 -- start start 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106570 0x7f1e6c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c116540 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1163c0 con 0x7f1e6c106ec0 2024-05-27T06:59:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.175+0000 7f1e71800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1166c0 con 0x7f1e6c106570 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48746/0 (socket says 172.21.15.16:48746) 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 -- 172.21.15.16:0/1876003241 learned_addr learned my addr 172.21.15.16:0/1876003241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6aa00700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106570 0x7f1e6c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6a000700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 -- 172.21.15.16:0/1876003241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106570 msgr2=0x7f1e6c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106570 0x7f1e6c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 -- 172.21.15.16:0/1876003241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 msgr2=0x7f1e6c19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6b400700 1 -- 172.21.15.16:0/1876003241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c1a1ec0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.176+0000 7f1e6a000700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e6b400700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007ee0 tx=0x7f1e6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400dbd0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e6400dd70 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c1a21b0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640058d0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.177+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1a9ad0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.178+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c0512b0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.181+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e6401a020 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.183+0000 7f1e63400700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 0x7f1e500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.183+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1e640e2470 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.183+0000 7f1e6aa00700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 0x7f1e500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.184+0000 7f1e6aa00700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 0x7f1e500b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c005950 tx=0x7f1e5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.185+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e64010070 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.431+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e6c02e5e0 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.435+0000 7f1e63400700 1 -- 172.21.15.16:0/1876003241 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24708 (secure 0 0 0) 0x7f1e640a9d80 con 0x7f1e6c10c5b0 2024-05-27T06:59:13.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:59:13.427 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:56:40.328992+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 msgr2=0x7f1e500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 0x7f1e500b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c005950 tx=0x7f1e5c00b410 comp rx=0 tx=0).stop 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 msgr2=0x7f1e6c1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007ee0 tx=0x7f1e6400d3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 shutdown_connections 2024-05-27T06:59:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1e500b2880 0x7f1e500b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e6c10c5b0 0x7f1e6c1a1820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6c106570 0x7f1e6c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 --2- 172.21.15.16:0/1876003241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e6c106ec0 0x7f1e6c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.438+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 >> 172.21.15.16:0/1876003241 conn(0x7f1e6c102070 msgr2=0x7f1e6c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.439+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 shutdown_connections 2024-05-27T06:59:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:13.439+0000 7f1e71800700 1 -- 172.21.15.16:0/1876003241 wait complete. 2024-05-27T06:59:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:13 smithi164 ceph-mon[35327]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:13 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1876003241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:59:13.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:13 smithi016 ceph-mon[31415]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:13 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1876003241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:59:13.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:13 smithi069 ceph-mon[35276]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:13.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:13 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1876003241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T06:59:14.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T06:59:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:15.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:15 smithi164 ceph-mon[35327]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:15.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:15 smithi016 ceph-mon[31415]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:15.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:15 smithi069 ceph-mon[35276]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.237+0000 7fcdcec00700 1 -- 172.21.15.16:0/3303745173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8077550 msgr2=0x7fcdc8110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.237+0000 7fcdcec00700 1 --2- 172.21.15.16:0/3303745173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8077550 0x7fcdc8110690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 -- 172.21.15.16:0/3303745173 shutdown_connections 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 --2- 172.21.15.16:0/3303745173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8110bd0 0x7fcdc8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 --2- 172.21.15.16:0/3303745173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8076c00 0x7fcdc8077010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 --2- 172.21.15.16:0/3303745173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8077550 0x7fcdc8110690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 -- 172.21.15.16:0/3303745173 >> 172.21.15.16:0/3303745173 conn(0x7fcdc80ffe90 msgr2=0x7fcdc81022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 -- 172.21.15.16:0/3303745173 shutdown_connections 2024-05-27T06:59:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.238+0000 7fcdcec00700 1 -- 172.21.15.16:0/3303745173 wait complete. 2024-05-27T06:59:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.239+0000 7fcdcec00700 1 Processor -- start 2024-05-27T06:59:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 -- start start 2024-05-27T06:59:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8077550 0x7fcdc819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8110bd0 0x7fcdc81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8116590 con 0x7fcdc8076c00 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8116410 con 0x7fcdc8077550 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdcec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8116710 con 0x7fcdc8110bd0 2024-05-27T06:59:17.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.240+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48758/0 (socket says 172.21.15.16:48758) 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 -- 172.21.15.16:0/437288694 learned_addr learned my addr 172.21.15.16:0/437288694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdbfe00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8077550 0x7fcdc819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdccc00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8110bd0 0x7fcdc81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 -- 172.21.15.16:0/437288694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8110bd0 msgr2=0x7fcdc81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8110bd0 0x7fcdc81a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 -- 172.21.15.16:0/437288694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8077550 msgr2=0x7fcdc819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8077550 0x7fcdc819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.241+0000 7fcdc7e00700 1 -- 172.21.15.16:0/437288694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc81a1e90 con 0x7fcdc8076c00 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdc7e00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800d950 tx=0x7fcdb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8016070 con 0x7fcdc8076c00 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc81a2180 con 0x7fcdc8076c00 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb8009dd0 con 0x7fcdc8076c00 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800f3c0 con 0x7fcdc8076c00 2024-05-27T06:59:17.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.242+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a9aa0 con 0x7fcdc8076c00 2024-05-27T06:59:17.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.244+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc80512b0 con 0x7fcdc8076c00 2024-05-27T06:59:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.246+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcdb8019070 con 0x7fcdc8076c00 2024-05-27T06:59:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.248+0000 7fcdc5600700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 0x7fcdac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.249+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcdb80e2290 con 0x7fcdc8076c00 2024-05-27T06:59:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.249+0000 7fcdbfe00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 0x7fcdac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:17.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.250+0000 7fcdbfe00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 0x7fcdac0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8103430 tx=0x7fcdb000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.254+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcdb80a9ba0 con 0x7fcdc8076c00 2024-05-27T06:59:17.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.572+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdc802e5e0 con 0x7fcdc8076c00 2024-05-27T06:59:17.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:17 smithi016 ceph-mon[31415]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:17.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:17 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:59:17.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.574+0000 7fcdc5600700 1 -- 172.21.15.16:0/437288694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7fcdb80a94a0 con 0x7fcdc8076c00 2024-05-27T06:59:17.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T06:59:17.566 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T06:59:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.577+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 msgr2=0x7fcdac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.577+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 0x7fcdac0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8103430 tx=0x7fcdb000b540 comp rx=0 tx=0).stop 2024-05-27T06:59:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.577+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 msgr2=0x7fcdc819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.577+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800d950 tx=0x7fcdb800dc60 comp rx=0 tx=0).stop 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 shutdown_connections 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fcdac0b2880 0x7fcdac0b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc8110bd0 0x7fcdc81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdc8076c00 0x7fcdc819cfc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 --2- 172.21.15.16:0/437288694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdc8077550 0x7fcdc819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 >> 172.21.15.16:0/437288694 conn(0x7fcdc80ffe90 msgr2=0x7fcdc8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 shutdown_connections 2024-05-27T06:59:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:17.578+0000 7fcdcec00700 1 -- 172.21.15.16:0/437288694 wait complete. 2024-05-27T06:59:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T06:59:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:17 smithi164 ceph-mon[35327]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:17 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:59:17.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:17 smithi069 ceph-mon[35276]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:17.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:17 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T06:59:18.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T06:55:13.518131+0000', 'modified': '2024-05-27T06:56:48.860392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34478, 'mds_2': 34451, 'mds_3': 34421, 'mds_4': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/1261534836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 1261534836}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 1261534836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/1178863608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 1178863608}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 1178863608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/1294360695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 1294360695}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 1294360695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2731773245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2731773245}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2731773245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/162118671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 162118671}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 162118671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.164:6839/3910557712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 3910557712}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 3910557712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34421': {'gid': 34421, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 41, 'addr': '172.21.15.69:6833/123964468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6832', 'nonce': 123964468}, {'type': 'v1', 'addr': '172.21.15.69:6833', 'nonce': 123964468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.69:6835/4223174404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 4223174404}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 4223174404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34478': {'gid': 34478, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.69:6837/2624187710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2624187710}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2624187710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.69:6839/962503578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 962503578}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 962503578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34517, 34478, 34451, 24481, 34421]} 2024-05-27T06:59:18.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T06:59:18.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T06:59:18.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:18 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/437288694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:18.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:18 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/437288694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:18.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:18 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/437288694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T06:59:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:19 smithi016 ceph-mon[31415]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:59:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:59:19.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:59:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:19 smithi164 ceph-mon[35327]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:59:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:59:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:59:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:19 smithi069 ceph-mon[35276]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T06:59:19.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T06:59:19.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T06:59:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.473+0000 7f5fe4400700 1 -- 172.21.15.16:0/1726853288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 msgr2=0x7f5fdc1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.473+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1726853288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1111c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400b0d0 tx=0x7f5fd400b3e0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 -- 172.21.15.16:0/1726853288 shutdown_connections 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1726853288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1111c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1726853288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc1045e0 0x7f5fdc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1726853288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc108f10 0x7f5fdc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 -- 172.21.15.16:0/1726853288 >> 172.21.15.16:0/1726853288 conn(0x7f5fdc100030 msgr2=0x7f5fdc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 -- 172.21.15.16:0/1726853288 shutdown_connections 2024-05-27T06:59:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.474+0000 7f5fe4400700 1 -- 172.21.15.16:0/1726853288 wait complete. 2024-05-27T06:59:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 Processor -- start 2024-05-27T06:59:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 -- start start 2024-05-27T06:59:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1045e0 0x7f5fdc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc108f10 0x7f5fdc19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114340 con 0x7f5fdc109860 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1141c0 con 0x7f5fdc1045e0 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.476+0000 7f5fe4400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1144c0 con 0x7f5fdc108f10 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48780/0 (socket says 172.21.15.16:48780) 2024-05-27T06:59:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 -- 172.21.15.16:0/1136363683 learned_addr learned my addr 172.21.15.16:0/1136363683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe1000700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc108f10 0x7f5fdc19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1045e0 0x7f5fdc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 -- 172.21.15.16:0/1136363683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc108f10 msgr2=0x7f5fdc19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc108f10 0x7f5fdc19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 -- 172.21.15.16:0/1136363683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1045e0 msgr2=0x7f5fdc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.477+0000 7f5fe2400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1045e0 0x7f5fdc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.478+0000 7f5fe2400700 1 -- 172.21.15.16:0/1136363683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1a1da0 con 0x7f5fdc109860 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.478+0000 7f5fe2400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009850 tx=0x7f5fd4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.478+0000 7f5fda000700 1 -- 172.21.15.16:0/1136363683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401f070 con 0x7f5fdc109860 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.478+0000 7f5fda000700 1 -- 172.21.15.16:0/1136363683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd4005500 con 0x7f5fdc109860 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.479+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1a2030 con 0x7f5fdc109860 2024-05-27T06:59:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.479+0000 7f5fda000700 1 -- 172.21.15.16:0/1136363683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401b8f0 con 0x7f5fdc109860 2024-05-27T06:59:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.479+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a2510 con 0x7f5fdc109860 2024-05-27T06:59:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.483+0000 7f5fda000700 1 -- 172.21.15.16:0/1136363683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5fd4022070 con 0x7f5fdc109860 2024-05-27T06:59:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5ff301ab80 1 Processor -- start 2024-05-27T06:59:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5ff301ab80 1 -- start start 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5fda000700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 0x7f5fc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.484+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640282dde20 0x5640282ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56402820bdf0 con 0x56402815edf0 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5640281fb990 con 0x56402824a9a0 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5640281ff9d0 con 0x5640282dde20 2024-05-27T06:59:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fda000700 1 -- 172.21.15.16:0/1136363683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5fd400f020 con 0x7f5fdc109860 2024-05-27T06:59:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38374/0 (socket says 172.21.15.16:38374) 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe1000700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 0x7f5fc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe1a00700 1 -- 172.21.15.16:0/707378746 learned_addr learned my addr 172.21.15.16:0/707378746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe2400700 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640282dde20 0x5640282ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.485+0000 7f5fe1000700 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 -- 172.21.15.16:0/707378746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640282dde20 msgr2=0x5640282ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640282dde20 0x5640282ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 -- 172.21.15.16:0/707378746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 msgr2=0x5640282dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 -- 172.21.15.16:0/707378746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640281c43d0 con 0x56402824a9a0 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1000700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 0x7f5fc40b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005e10 tx=0x7f5fc800bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.486+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000bd50 tx=0x7f5fd00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab59313e2c28c7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.487+0000 7f5fcea00700 1 -- 172.21.15.16:0/707378746 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0010e10 con 0x56402824a9a0 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.487+0000 7f5fcea00700 1 -- 172.21.15.16:0/707378746 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd0004cc0 con 0x56402824a9a0 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.487+0000 7f5fcea00700 1 -- 172.21.15.16:0/707378746 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011440 con 0x56402824a9a0 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 -- 172.21.15.16:0/707378746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 msgr2=0x5640282dd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000bd50 tx=0x7f5fd00106b0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 -- 172.21.15.16:0/707378746 shutdown_connections 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5640282dde20 0x5640282ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282dd2e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/707378746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 -- 172.21.15.16:0/707378746 >> 172.21.15.16:0/707378746 conn(0x5640281f8340 msgr2=0x5640282d9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 -- 172.21.15.16:0/707378746 shutdown_connections 2024-05-27T06:59:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.488+0000 7f5ff301ab80 1 -- 172.21.15.16:0/707378746 wait complete. 2024-05-27T06:59:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.489+0000 7f5ff301ab80 1 -- start start 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.489+0000 7f5ff301ab80 1 Processor -- start 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564028101f50 con 0x56402824a9a0 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56402819c710 con 0x5640282dde20 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5640282eab30 con 0x56402815edf0 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5ff301ab80 10 client.0 _ll_register_callbacks cb 0x5640281a5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5fe1000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5fe1000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48800/0 (socket says 172.21.15.16:48800) 2024-05-27T06:59:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5fe1000700 1 -- 172.21.15.16:0/2090768670 learned_addr learned my addr 172.21.15.16:0/2090768670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.490+0000 7f5fe2400700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 -- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 msgr2=0x5640282e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 -- 172.21.15.16:0/2090768670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 msgr2=0x5640282e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 -- 172.21.15.16:0/2090768670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402814b880 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fe1000700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc06c670 tx=0x7f5fc800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cee6f2f9ff6dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.491+0000 7f5fc3400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8010dd0 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd0004cc0 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5fc801b040 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8032050 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5ff301ab80 10 client.15051 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5640281cc010 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.492+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd0004cc0 con 0x56402824a9a0 2024-05-27T06:59:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5fc801f070 con 0x56402824a9a0 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 10 client.15051 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 20 client.15051 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5640282f7b00 0x5640282f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 4 client.15051 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.493+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402820dd40 con 0x5640282f7b00 2024-05-27T06:59:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.494+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5640282f7b00 0x5640282f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.495+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5640282f7b00 0x5640282f8f70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c76bd57ed36ecf1c server_cookie=595bc67dcb1b3cae in_seq=0 out_seq=0 2024-05-27T06:59:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.495+0000 7f5fc3400700 10 client.15051 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T06:59:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5fc3400700 1 -- 172.21.15.16:0/2090768670 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56402820dd40 con 0x5640282f7b00 2024-05-27T06:59:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5fc3400700 10 client.15051 handle_command_reply: tid=0 2024-05-27T06:59:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 client.15051 shutdown 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 msgr2=0x5640282e7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc06c670 tx=0x7f5fc800b410 comp rx=0 tx=0).stop 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 shutdown_connections 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5640282f7b00 0x5640282f8f70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/2090768670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402824a9a0 0x5640282e7b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5fc3400700 0 client.15051 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T06:59:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.497+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 >> 172.21.15.16:0/2090768670 conn(0x5640281f8340 msgr2=0x5640282dd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.498+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 shutdown_connections 2024-05-27T06:59:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.498+0000 7f5ff301ab80 1 -- 172.21.15.16:0/2090768670 wait complete. 2024-05-27T06:59:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.498+0000 7f5ff301ab80 20 client.15051 trim_cache size 0 max 16384 2024-05-27T06:59:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.521+0000 7f5ff301ab80 1 Processor -- start 2024-05-27T06:59:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.521+0000 7f5ff301ab80 1 -- start start 2024-05-27T06:59:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 0x5640282d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640282f72b0 con 0x5640282dde20 2024-05-27T06:59:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5640282f7980 con 0x56402824a9a0 2024-05-27T06:59:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5640282f8050 con 0x56402815edf0 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.522+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 0x5640282d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47644/0 (socket says 172.21.15.16:47644) 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe1a00700 1 -- 172.21.15.16:0/518389342 learned_addr learned my addr 172.21.15.16:0/518389342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe1000700 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 -- 172.21.15.16:0/518389342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 msgr2=0x5640282e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 -- 172.21.15.16:0/518389342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 msgr2=0x5640282e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 -- 172.21.15.16:0/518389342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640282e8440 con 0x5640282dde20 2024-05-27T06:59:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.523+0000 7f5fe2400700 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 0x5640282d8f00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40e10c0 tx=0x7f5fd40aa4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e72e2b49770b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.524+0000 7f5fce000700 1 -- 172.21.15.16:0/518389342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40a6400 con 0x5640282dde20 2024-05-27T06:59:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.524+0000 7f5fce000700 1 -- 172.21.15.16:0/518389342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd40c4070 con 0x5640282dde20 2024-05-27T06:59:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.524+0000 7f5fce000700 1 -- 172.21.15.16:0/518389342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40aac70 con 0x5640282dde20 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.524+0000 7f5ff301ab80 1 -- 172.21.15.16:0/518389342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 msgr2=0x5640282d8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.524+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 0x5640282d8f00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40e10c0 tx=0x7f5fd40aa4e0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 -- 172.21.15.16:0/518389342 shutdown_connections 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640282dde20 0x5640282d8f00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402815edf0 0x5640282e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/518389342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56402824a9a0 0x5640282e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 -- 172.21.15.16:0/518389342 >> 172.21.15.16:0/518389342 conn(0x5640281f8340 msgr2=0x5640282d8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 -- 172.21.15.16:0/518389342 shutdown_connections 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 -- 172.21.15.16:0/518389342 wait complete. 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.525+0000 7f5ff301ab80 1 -- start start 2024-05-27T06:59:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 Processor -- start 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402824a9a0 0x5640282e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640282f8df0 con 0x56402815edf0 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5640282f8720 con 0x5640282dde20 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5640282f94c0 con 0x56402824a9a0 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5ff301ab80 10 client.0 _ll_register_callbacks cb 0x5640281a5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.526+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48814/0 (socket says 172.21.15.16:48814) 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 -- 172.21.15.16:0/538480103 learned_addr learned my addr 172.21.15.16:0/538480103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1000700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402824a9a0 0x5640282e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe2400700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 -- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402824a9a0 msgr2=0x5640282e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402824a9a0 0x5640282e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 -- 172.21.15.16:0/538480103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 msgr2=0x5640282e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 -- 172.21.15.16:0/538480103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564028301510 con 0x56402815edf0 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fe1a00700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5640282dffa0 tx=0x7f5fd0010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401456146dee661b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.527+0000 7f5fc3400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00055e0 con 0x56402815edf0 2024-05-27T06:59:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5fd0004710 con 0x56402815edf0 2024-05-27T06:59:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd40c4070 con 0x56402815edf0 2024-05-27T06:59:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fd0039050 con 0x56402815edf0 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5ff301ab80 10 client.15057 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.528+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564028302770 con 0x56402815edf0 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd40c4070 con 0x56402815edf0 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5fd00287f0 con 0x56402815edf0 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 10 client.15057 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 20 client.15057 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564028364b00 0x564028365f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 4 client.15057 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.529+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "status"}) -- 0x564028207520 con 0x564028364b00 2024-05-27T06:59:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.530+0000 7f5fe1000700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564028364b00 0x564028365f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.531+0000 7f5fe1000700 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564028364b00 0x564028365f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=26f5ab15f990619f server_cookie=912e513caaf7d173 in_seq=0 out_seq=0 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.531+0000 7f5fc3400700 10 client.15057 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.531+0000 7f5fc3400700 1 -- 172.21.15.16:0/538480103 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564028207520 con 0x564028364b00 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.531+0000 7f5fc3400700 10 client.15057 handle_command_reply: tid=0 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.531+0000 7f5ff301ab80 1 client.15057 shutdown 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 msgr2=0x5640282e15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5640282dffa0 tx=0x7f5fd0010e60 comp rx=0 tx=0).stop 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 shutdown_connections 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564028364b00 0x564028365f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640282dde20 0x5640282e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56402815edf0 0x5640282e15b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 --2- 172.21.15.16:0/538480103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402824a9a0 0x5640282e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 >> 172.21.15.16:0/538480103 conn(0x5640281f8340 msgr2=0x5640282d8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5fc3400700 0 client.15057 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 shutdown_connections 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 1 -- 172.21.15.16:0/538480103 wait complete. 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.532+0000 7f5ff301ab80 20 client.15057 trim_cache size 0 max 16384 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:59:21.525 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 159.12142713599999, 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 286.89518944399998 2024-05-27T06:59:21.526 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 msgr2=0x7f5fc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 0x7f5fc40b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005e10 tx=0x7f5fc800bb70 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 msgr2=0x7f5fdc1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009850 tx=0x7f5fd4007d90 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 shutdown_connections 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5fc40b2880 0x7f5fc40b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fdc109860 0x7f5fdc1a1700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1045e0 0x7f5fdc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 --2- 172.21.15.16:0/1136363683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc108f10 0x7f5fdc19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 >> 172.21.15.16:0/1136363683 conn(0x7f5fdc100030 msgr2=0x7f5fdc110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 shutdown_connections 2024-05-27T06:59:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:21.533+0000 7f5fe4400700 1 -- 172.21.15.16:0/1136363683 wait complete. 2024-05-27T06:59:21.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:21 smithi164 ceph-mon[35327]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:21.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:21 smithi016 ceph-mon[31415]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:21.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:21 smithi069 ceph-mon[35276]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:22.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-05-27T06:59:22.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:22 smithi164 ceph-mon[35327]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:22.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:22 smithi016 ceph-mon[31415]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:22.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:22 smithi069 ceph-mon[35276]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:25.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 -- 172.21.15.16:0/1106746967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4076670 msgr2=0x7f64c4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 --2- 172.21.15.16:0/1106746967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4076670 0x7f64c4110650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009990 tx=0x7f64ac009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 -- 172.21.15.16:0/1106746967 shutdown_connections 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 --2- 172.21.15.16:0/1106746967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4110b90 0x7f64c4112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 --2- 172.21.15.16:0/1106746967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4075d20 0x7f64c4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 --2- 172.21.15.16:0/1106746967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4076670 0x7f64c4110650 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.432+0000 7f64cb800700 1 -- 172.21.15.16:0/1106746967 >> 172.21.15.16:0/1106746967 conn(0x7f64c4100030 msgr2=0x7f64c4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.433+0000 7f64cb800700 1 -- 172.21.15.16:0/1106746967 shutdown_connections 2024-05-27T06:59:25.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.433+0000 7f64cb800700 1 -- 172.21.15.16:0/1106746967 wait complete. 2024-05-27T06:59:25.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:25 smithi016 ceph-mon[31415]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:25.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.434+0000 7f64cb800700 1 Processor -- start 2024-05-27T06:59:25.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.434+0000 7f64cb800700 1 -- start start 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4076670 0x7f64c419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 0x7f64c41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4116590 con 0x7f64c4075d20 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4116410 con 0x7f64c4110b90 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64cb800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4116710 con 0x7f64c4076670 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42676/0 (socket says 172.21.15.16:42676) 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64c8e00700 1 -- 172.21.15.16:0/2168821690 learned_addr learned my addr 172.21.15.16:0/2168821690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64bbe00700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4076670 0x7f64c419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.435+0000 7f64c9800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 0x7f64c41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 -- 172.21.15.16:0/2168821690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4076670 msgr2=0x7f64c419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4076670 0x7f64c419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 -- 172.21.15.16:0/2168821690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 msgr2=0x7f64c41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 0x7f64c41a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 -- 172.21.15.16:0/2168821690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41a1e00 con 0x7f64c4075d20 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c9800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 0x7f64c41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c9b0 tx=0x7f64b400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c2000700 1 -- 172.21.15.16:0/2168821690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4007b00 con 0x7f64c4075d20 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c2000700 1 -- 172.21.15.16:0/2168821690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64b4007ca0 con 0x7f64c4075d20 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c41a20f0 con 0x7f64c4075d20 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.436+0000 7f64c2000700 1 -- 172.21.15.16:0/2168821690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4018660 con 0x7f64c4075d20 2024-05-27T06:59:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.437+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c41a2630 con 0x7f64c4075d20 2024-05-27T06:59:25.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.441+0000 7f64c2000700 1 -- 172.21.15.16:0/2168821690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f64b400f450 con 0x7f64c4075d20 2024-05-27T06:59:25.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 Processor -- start 2024-05-27T06:59:25.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 -- start start 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146389a0 0x55ca146cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca145f9df0 con 0x55ca146cbe20 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca145e9990 con 0x55ca146389a0 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.442+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca145ed9d0 con 0x55ca1454cdf0 2024-05-27T06:59:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42686/0 (socket says 172.21.15.16:42686) 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 -- 172.21.15.16:0/1646056314 learned_addr learned my addr 172.21.15.16:0/1646056314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64c9800700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146389a0 0x55ca146cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64c8e00700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 -- 172.21.15.16:0/1646056314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 msgr2=0x55ca146cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 -- 172.21.15.16:0/1646056314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146389a0 msgr2=0x55ca146cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146389a0 0x55ca146cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64c2000700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 0x7f64a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64bbe00700 1 -- 172.21.15.16:0/1646056314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca145b23d0 con 0x55ca146cbe20 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.443+0000 7f64c8e00700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.444+0000 7f64c2000700 1 -- 172.21.15.16:0/2168821690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f64b40e26f0 con 0x7f64c4075d20 2024-05-27T06:59:25.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.444+0000 7f64c9800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 0x7f64a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.444+0000 7f64bbe00700 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64ac015000 tx=0x7f64ac003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee61bcf0ec154f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.444+0000 7f64ba000700 1 -- 172.21.15.16:0/1646056314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac026070 con 0x55ca146cbe20 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.445+0000 7f64ba000700 1 -- 172.21.15.16:0/1646056314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64ac003d10 con 0x55ca146cbe20 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.445+0000 7f64ba000700 1 -- 172.21.15.16:0/1646056314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0056e0 con 0x55ca146cbe20 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.445+0000 7f64da3d6b80 1 -- 172.21.15.16:0/1646056314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 msgr2=0x55ca146d80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64ac015000 tx=0x7f64ac003720 comp rx=0 tx=0).stop 2024-05-27T06:59:25.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64c9800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 0x7f64a80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0094f0 tx=0x7f64bc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 -- 172.21.15.16:0/1646056314 shutdown_connections 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146d80c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146389a0 0x55ca146cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/1646056314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 -- 172.21.15.16:0/1646056314 >> 172.21.15.16:0/1646056314 conn(0x55ca145e6340 msgr2=0x55ca146c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 -- 172.21.15.16:0/1646056314 shutdown_connections 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 -- 172.21.15.16:0/1646056314 wait complete. 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.446+0000 7f64da3d6b80 1 -- start start 2024-05-27T06:59:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 Processor -- start 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146cbe20 0x55ca146d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca144eff50 con 0x55ca146389a0 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca1458a710 con 0x55ca1454cdf0 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.447+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca146d8ad0 con 0x55ca146cbe20 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64da3d6b80 10 client.0 _ll_register_callbacks cb 0x55ca14593c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42694/0 (socket says 172.21.15.16:42694) 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 -- 172.21.15.16:0/2008080961 learned_addr learned my addr 172.21.15.16:0/2008080961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64c9800700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146cbe20 0x55ca146d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 -- 172.21.15.16:0/2008080961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146cbe20 msgr2=0x55ca146d60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146cbe20 0x55ca146d60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 -- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 msgr2=0x55ca146d5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.448+0000 7f64bbe00700 1 -- 172.21.15.16:0/2008080961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca14539880 con 0x55ca146389a0 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64bbe00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009750 tx=0x7f64ac003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63affa8fde178a7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac026070 con 0x55ca146389a0 2024-05-27T06:59:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f64ac00b9e0 con 0x55ca146389a0 2024-05-27T06:59:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ac003d10 con 0x55ca146389a0 2024-05-27T06:59:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64ac00bbe0 con 0x55ca146389a0 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64da3d6b80 10 client.15069 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.449+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca145ba010 con 0x55ca146389a0 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64ac003d10 con 0x55ca146389a0 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f64ac00fc90 con 0x55ca146389a0 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 10 client.15069 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 10 client.15069 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 20 client.15069 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca146e5560 0x55ca146e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 4 client.15069 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.450+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca145fbd40 con 0x55ca146e5560 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.451+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca146e5560 0x55ca146e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.452+0000 7f64c8e00700 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca146e5560 0x55ca146e69d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dbac3f60243c2537 server_cookie=e823f96283073e3a in_seq=0 out_seq=0 2024-05-27T06:59:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.452+0000 7f64b2a00700 10 client.15069 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T06:59:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.454+0000 7f64b2a00700 1 -- 172.21.15.16:0/2008080961 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ca145fbd40 con 0x55ca146e5560 2024-05-27T06:59:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.454+0000 7f64b2a00700 10 client.15069 handle_command_reply: tid=0 2024-05-27T06:59:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 client.15069 shutdown 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 msgr2=0x55ca146d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009750 tx=0x7f64ac003f20 comp rx=0 tx=0).stop 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 shutdown_connections 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146cbe20 0x55ca146d60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca146e5560 0x55ca146e69d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/2008080961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d5b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 >> 172.21.15.16:0/2008080961 conn(0x55ca145e6340 msgr2=0x55ca146cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64b2a00700 0 client.15069 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 shutdown_connections 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 1 -- 172.21.15.16:0/2008080961 wait complete. 2024-05-27T06:59:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.455+0000 7f64da3d6b80 20 client.15069 trim_cache size 0 max 16384 2024-05-27T06:59:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 Processor -- start 2024-05-27T06:59:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 -- start start 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 0x55ca146cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e4d10 con 0x55ca146cbe20 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e53e0 con 0x55ca1454cdf0 2024-05-27T06:59:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.479+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e5ab0 con 0x55ca146389a0 2024-05-27T06:59:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42708/0 (socket says 172.21.15.16:42708) 2024-05-27T06:59:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 -- 172.21.15.16:0/4239269594 learned_addr learned my addr 172.21.15.16:0/4239269594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64bbe00700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 0x55ca146cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c8e00700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 -- 172.21.15.16:0/4239269594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 msgr2=0x55ca146cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 0x55ca146cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 -- 172.21.15.16:0/4239269594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 msgr2=0x55ca146d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64c9800700 1 -- 172.21.15.16:0/4239269594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca146d6440 con 0x55ca146cbe20 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.480+0000 7f64bbe00700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 0x55ca146cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.481+0000 7f64c9800700 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64bc016580 tx=0x7f64bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd9d8d2b0efc6f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.481+0000 7f64b9600700 1 -- 172.21.15.16:0/4239269594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc01d030 con 0x55ca146cbe20 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.481+0000 7f64b9600700 1 -- 172.21.15.16:0/4239269594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64bc028070 con 0x55ca146cbe20 2024-05-27T06:59:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.481+0000 7f64b9600700 1 -- 172.21.15.16:0/4239269594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc017400 con 0x55ca146cbe20 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 -- 172.21.15.16:0/4239269594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 msgr2=0x55ca146cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64bc016580 tx=0x7f64bc014000 comp rx=0 tx=0).stop 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 -- 172.21.15.16:0/4239269594 shutdown_connections 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146cbe20 0x55ca146cce80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca1454cdf0 0x55ca146d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/4239269594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca146389a0 0x55ca146cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 -- 172.21.15.16:0/4239269594 >> 172.21.15.16:0/4239269594 conn(0x55ca145e6340 msgr2=0x55ca146cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 -- 172.21.15.16:0/4239269594 shutdown_connections 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.482+0000 7f64da3d6b80 1 -- 172.21.15.16:0/4239269594 wait complete. 2024-05-27T06:59:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 -- start start 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 Processor -- start 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 0x55ca146eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e6850 con 0x55ca146389a0 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e6180 con 0x55ca146cbe20 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca146e6f20 con 0x55ca1454cdf0 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.483+0000 7f64da3d6b80 10 client.0 _ll_register_callbacks cb 0x55ca14593c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42718/0 (socket says 172.21.15.16:42718) 2024-05-27T06:59:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 -- 172.21.15.16:0/240298840 learned_addr learned my addr 172.21.15.16:0/240298840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64c8e00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64c9800700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 0x55ca146eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 -- 172.21.15.16:0/240298840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 msgr2=0x55ca146cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 -- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 msgr2=0x55ca146eee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 0x55ca146eee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 -- 172.21.15.16:0/240298840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca146ef3d0 con 0x55ca146389a0 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64c8e00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:59:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64c9800700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 0x55ca146eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64bbe00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0098e0 tx=0x7f64ac0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1beb7df596481a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.484+0000 7f64b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac026040 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f64ac00b9e0 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc028070 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64ac040050 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64da3d6b80 10 client.15075 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca146f0630 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.485+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64bc028070 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f64ac00f980 con 0x55ca146389a0 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 10 client.15075 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 20 client.15075 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca14752b70 0x55ca14753fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 4 client.15075 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:59:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.486+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "status"}) -- 0x55ca145f5520 con 0x55ca14752b70 2024-05-27T06:59:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.487+0000 7f64c8e00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca14752b70 0x55ca14753fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.488+0000 7f64c8e00700 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca14752b70 0x55ca14753fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3724d77bcaeb79b5 server_cookie=2ad0611e81ef5d88 in_seq=0 out_seq=0 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.488+0000 7f64b2a00700 10 client.15075 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.488+0000 7f64b2a00700 1 -- 172.21.15.16:0/240298840 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ca145f5520 con 0x55ca14752b70 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.488+0000 7f64b2a00700 10 client.15075 handle_command_reply: tid=0 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.488+0000 7f64da3d6b80 1 client.15075 shutdown 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 msgr2=0x55ca146d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0098e0 tx=0x7f64ac0036f0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 shutdown_connections 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ca14752b70 0x55ca14753fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca146cbe20 0x55ca146eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca1454cdf0 0x55ca146cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 --2- 172.21.15.16:0/240298840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca146389a0 0x55ca146d0080 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64b2a00700 0 client.15075 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 >> 172.21.15.16:0/240298840 conn(0x55ca145e6340 msgr2=0x55ca146cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 shutdown_connections 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 1 -- 172.21.15.16:0/240298840 wait complete. 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.489+0000 7f64da3d6b80 20 client.15075 trim_cache size 0 max 16384 2024-05-27T06:59:25.482 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34421, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 164.09645938, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 322.29220028899999 2024-05-27T06:59:25.483 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 msgr2=0x7f64a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 0x7f64a80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0094f0 tx=0x7f64bc009040 comp rx=0 tx=0).stop 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 msgr2=0x7f64c419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c9b0 tx=0x7f64b400cd70 comp rx=0 tx=0).stop 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 shutdown_connections 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f64a80b2880 0x7f64a80b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c4110b90 0x7f64c41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4075d20 0x7f64c419cfe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 --2- 172.21.15.16:0/2168821690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c4076670 0x7f64c419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 >> 172.21.15.16:0/2168821690 conn(0x7f64c4100030 msgr2=0x7f64c4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 shutdown_connections 2024-05-27T06:59:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:25.490+0000 7f64cb800700 1 -- 172.21.15.16:0/2168821690 wait complete. 2024-05-27T06:59:25.535 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:25 smithi164 ceph-mon[35327]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:25.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:25 smithi069 ceph-mon[35276]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:26.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T06:59:27.050 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:27.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:27 smithi016 ceph-mon[31415]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:27.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:27 smithi164 ceph-mon[35327]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:27.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:27 smithi069 ceph-mon[35276]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:29.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:29 smithi016 ceph-mon[31415]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:29.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:29 smithi164 ceph-mon[35327]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:29.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:29 smithi069 ceph-mon[35276]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.635+0000 7f9fbfc00700 1 -- 172.21.15.16:0/3444847876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 msgr2=0x7f9fb8113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.635+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/3444847876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb8113340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b0a0 tx=0x7f9fb000b3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 -- 172.21.15.16:0/3444847876 shutdown_connections 2024-05-27T06:59:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/3444847876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb8113340 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/3444847876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/3444847876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 0x7f9fb8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 -- 172.21.15.16:0/3444847876 >> 172.21.15.16:0/3444847876 conn(0x7f9fb8102040 msgr2=0x7f9fb8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 -- 172.21.15.16:0/3444847876 shutdown_connections 2024-05-27T06:59:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.636+0000 7f9fbfc00700 1 -- 172.21.15.16:0/3444847876 wait complete. 2024-05-27T06:59:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.637+0000 7f9fbfc00700 1 Processor -- start 2024-05-27T06:59:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.637+0000 7f9fbfc00700 1 -- start start 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 0x7f9fb819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81165d0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8116450 con 0x7f9fb810c5a0 2024-05-27T06:59:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbfc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8116750 con 0x7f9fb8106560 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fb7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fb7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42744/0 (socket says 172.21.15.16:42744) 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fb7e00700 1 -- 172.21.15.16:0/2508627514 learned_addr learned my addr 172.21.15.16:0/2508627514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbd200700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 0x7f9fb819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.638+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 -- 172.21.15.16:0/2508627514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 msgr2=0x7f9fb819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 0x7f9fb819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 -- 172.21.15.16:0/2508627514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 msgr2=0x7f9fb81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 -- 172.21.15.16:0/2508627514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81a1db0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.639+0000 7f9fb5600700 1 -- 172.21.15.16:0/2508627514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4017070 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.640+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb81a20a0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.640+0000 7f9fb5600700 1 -- 172.21.15.16:0/2508627514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa4009dc0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.640+0000 7f9fb5600700 1 -- 172.21.15.16:0/2508627514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400f3c0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.640+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a99c0 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.644+0000 7f9fb5600700 1 -- 172.21.15.16:0/2508627514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9fa401a070 con 0x7f9fb8106eb0 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.645+0000 7f9fce6dcb80 1 Processor -- start 2024-05-27T06:59:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.645+0000 7f9fce6dcb80 1 -- start start 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fb5600700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 0x7f9fa00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db1692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559dafeadf0 0x5559db169820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 0x5559db1760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559db097df0 con 0x5559db169e20 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559db087990 con 0x5559db0d69a0 2024-05-27T06:59:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559db08b9d0 con 0x5559dafeadf0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fb5600700 1 -- 172.21.15.16:0/2508627514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9fa40e1f00 con 0x7f9fb8106eb0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 0x7f9fa00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db1692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fbdc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 0x5559db1760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db1692e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32828/0 (socket says 172.21.15.16:32828) 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fbd200700 1 -- 172.21.15.16:0/2675049574 learned_addr learned my addr 172.21.15.16:0/2675049574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.646+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559dafeadf0 0x5559db169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 -- 172.21.15.16:0/2675049574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559dafeadf0 msgr2=0x5559db169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559dafeadf0 0x5559db169820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 -- 172.21.15.16:0/2675049574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 msgr2=0x5559db1692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db1692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 -- 172.21.15.16:0/2675049574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559db0503d0 con 0x5559db169e20 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 0x7f9fa00b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b830 tx=0x7f9fb0009e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.647+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 0x5559db1760c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b070 tx=0x7f9fb00095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f22e119995023e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.648+0000 7f9faa000700 1 -- 172.21.15.16:0/2675049574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009bc0 con 0x5559db169e20 2024-05-27T06:59:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.648+0000 7f9faa000700 1 -- 172.21.15.16:0/2675049574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fb0007d30 con 0x5559db169e20 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.648+0000 7f9faa000700 1 -- 172.21.15.16:0/2675049574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0020d30 con 0x5559db169e20 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/2675049574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 msgr2=0x5559db1760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 0x5559db1760c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b070 tx=0x7f9fb00095d0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/2675049574 shutdown_connections 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559db169e20 0x5559db1760c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db1692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/2675049574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559dafeadf0 0x5559db169820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/2675049574 >> 172.21.15.16:0/2675049574 conn(0x5559db084340 msgr2=0x5559db165470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/2675049574 shutdown_connections 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.649+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/2675049574 wait complete. 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.650+0000 7f9fce6dcb80 1 -- start start 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.650+0000 7f9fce6dcb80 1 Processor -- start 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db173bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db1740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559daf8df50 con 0x5559dafeadf0 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559db028710 con 0x5559db169e20 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559db176ad0 con 0x5559db0d69a0 2024-05-27T06:59:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fce6dcb80 10 client.0 _ll_register_callbacks cb 0x5559db031c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42764/0 (socket says 172.21.15.16:42764) 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fbd200700 1 -- 172.21.15.16:0/1050273210 learned_addr learned my addr 172.21.15.16:0/1050273210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db173bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.651+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db1740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 -- 172.21.15.16:0/1050273210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 msgr2=0x5559db173bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db173bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 -- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 msgr2=0x5559db1740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db1740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 -- 172.21.15.16:0/1050273210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559dafd7880 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbdc00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db1740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00bb70 tx=0x7f9fac00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4299cffff5ea7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9f9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac003d60 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9fac004830 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.652+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb0007d30 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.653+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fac024050 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.653+0000 7f9fce6dcb80 10 client.15087 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.653+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559db058010 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.653+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fb0007d30 con 0x5559dafeadf0 2024-05-27T06:59:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.653+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fac011440 con 0x5559dafeadf0 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 10 client.15087 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 10 client.15087 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 20 client.15087 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db183b40 0x5559db184fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 4 client.15087 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559db099d40 con 0x5559db183b40 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.654+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db183b40 0x5559db184fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.655+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db183b40 0x5559db184fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=584ff4d07164d6f4 server_cookie=976ceaba819bb047 in_seq=0 out_seq=0 2024-05-27T06:59:29.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.655+0000 7f9f9ea00700 10 client.15087 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.658+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1050273210 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5559db099d40 con 0x5559db183b40 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.658+0000 7f9f9ea00700 10 client.15087 handle_command_reply: tid=0 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.658+0000 7f9fce6dcb80 1 client.15087 shutdown 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 msgr2=0x5559db173670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00bb70 tx=0x7f9fac00bf30 comp rx=0 tx=0).stop 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 shutdown_connections 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db183b40 0x5559db184fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db1740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db173670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1050273210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9f9ea00700 0 client.15087 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T06:59:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 >> 172.21.15.16:0/1050273210 conn(0x5559db084340 msgr2=0x5559db1692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 shutdown_connections 2024-05-27T06:59:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1050273210 wait complete. 2024-05-27T06:59:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.659+0000 7f9fce6dcb80 20 client.15087 trim_cache size 0 max 16384 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.683+0000 7f9fce6dcb80 1 Processor -- start 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.683+0000 7f9fce6dcb80 1 -- start start 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db16ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db169060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559db1832f0 con 0x5559dafeadf0 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559db1839c0 con 0x5559db169e20 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559db184090 con 0x5559db0d69a0 2024-05-27T06:59:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fbdc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db169060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fb7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db16ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42780/0 (socket says 172.21.15.16:42780) 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.684+0000 7f9fbd200700 1 -- 172.21.15.16:0/3176679094 learned_addr learned my addr 172.21.15.16:0/3176679094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 -- 172.21.15.16:0/3176679094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 msgr2=0x5559db16ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db16ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 -- 172.21.15.16:0/3176679094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 msgr2=0x5559db169060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db169060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 -- 172.21.15.16:0/3176679094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559db176410 con 0x5559dafeadf0 2024-05-27T06:59:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.685+0000 7f9fbd200700 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011890 tx=0x7f9fac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2168c9a4491eee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.686+0000 7f9fa9600700 1 -- 172.21.15.16:0/3176679094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac003e60 con 0x5559dafeadf0 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.686+0000 7f9fa9600700 1 -- 172.21.15.16:0/3176679094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fac0054c0 con 0x5559dafeadf0 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.686+0000 7f9fa9600700 1 -- 172.21.15.16:0/3176679094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004690 con 0x5559dafeadf0 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.686+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/3176679094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 msgr2=0x5559db165610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.686+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011890 tx=0x7f9fac005780 comp rx=0 tx=0).stop 2024-05-27T06:59:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/3176679094 shutdown_connections 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db169e20 0x5559db169060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db165610 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/3176679094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db0d69a0 0x5559db16ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/3176679094 >> 172.21.15.16:0/3176679094 conn(0x5559db084340 msgr2=0x5559db164f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/3176679094 shutdown_connections 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/3176679094 wait complete. 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 -- start start 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.687+0000 7f9fce6dcb80 1 Processor -- start 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db187eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db169e20 0x5559db1884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559db184e30 con 0x5559dafeadf0 2024-05-27T06:59:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559db184760 con 0x5559db0d69a0 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559db185500 con 0x5559db169e20 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fce6dcb80 10 client.0 _ll_register_callbacks cb 0x5559db031c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fbd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42790/0 (socket says 172.21.15.16:42790) 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.688+0000 7f9fbd200700 1 -- 172.21.15.16:0/1421990176 learned_addr learned my addr 172.21.15.16:0/1421990176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db187eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 -- 172.21.15.16:0/1421990176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db169e20 msgr2=0x5559db1884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db169e20 0x5559db1884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 -- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 msgr2=0x5559db187eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db187eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 -- 172.21.15.16:0/1421990176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559db188ca0 con 0x5559dafeadf0 2024-05-27T06:59:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9fbd200700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0116f0 tx=0x7f9fac011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6426c19048425299 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9f9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.689+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac036070 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9fac021040 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac0054c0 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fac042050 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9fce6dcb80 10 client.15093 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559db1f7ae0 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fac0054c0 con 0x5559dafeadf0 2024-05-27T06:59:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fac01bcb0 con 0x5559dafeadf0 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.690+0000 7f9fce6dcb80 10 client.15093 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fce6dcb80 10 client.15093 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fce6dcb80 20 client.15093 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db1f9b70 0x5559db1fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fce6dcb80 4 client.15093 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "status"}) -- 0x5559db093520 con 0x5559db1f9b70 2024-05-27T06:59:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.691+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db1f9b70 0x5559db1fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.692+0000 7f9fb7e00700 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db1f9b70 0x5559db1fafe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62308dc6ba973bd7 server_cookie=c9479d49d43372b2 in_seq=0 out_seq=0 2024-05-27T06:59:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.692+0000 7f9f9ea00700 10 client.15093 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T06:59:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9f9ea00700 1 -- 172.21.15.16:0/1421990176 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5559db093520 con 0x5559db1f9b70 2024-05-27T06:59:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9f9ea00700 10 client.15093 handle_command_reply: tid=0 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 client.15093 shutdown 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 msgr2=0x5559db18eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0116f0 tx=0x7f9fac011720 comp rx=0 tx=0).stop 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 shutdown_connections 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5559db1f9b70 0x5559db1fafe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559db169e20 0x5559db1884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559dafeadf0 0x5559db18eb70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 --2- 172.21.15.16:0/1421990176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559db0d69a0 0x5559db187eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 >> 172.21.15.16:0/1421990176 conn(0x5559db084340 msgr2=0x5559db169060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9f9ea00700 0 client.15093 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 shutdown_connections 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.693+0000 7f9fce6dcb80 1 -- 172.21.15.16:0/1421990176 wait complete. 2024-05-27T06:59:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fce6dcb80 20 client.15093 trim_cache size 0 max 16384 2024-05-27T06:59:29.692 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:59:29.692 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:59:29.692 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-05-27T06:59:29.692 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34451, 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 169.31774404500001, 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 311.96732428199999 2024-05-27T06:59:29.693 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 msgr2=0x7f9fa00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 0x7f9fa00b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b830 tx=0x7f9fb0009e20 comp rx=0 tx=0).stop 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 msgr2=0x7f9fb819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400dc60 comp rx=0 tx=0).stop 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.694+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 shutdown_connections 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f9fa00b2880 0x7f9fa00b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fb810c5a0 0x7f9fb81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fb8106560 0x7f9fb819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 --2- 172.21.15.16:0/2508627514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fb8106eb0 0x7f9fb819d480 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 >> 172.21.15.16:0/2508627514 conn(0x7f9fb8102040 msgr2=0x7f9fb8103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 shutdown_connections 2024-05-27T06:59:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:29.695+0000 7f9fbfc00700 1 -- 172.21.15.16:0/2508627514 wait complete. 2024-05-27T06:59:30.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T06:59:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:31.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:31 smithi164 ceph-mon[35327]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:31.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:31 smithi016 ceph-mon[31415]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:31.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:31 smithi069 ceph-mon[35276]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 -- 172.21.15.16:0/1976836032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 msgr2=0x7f88c4117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 --2- 172.21.15.16:0/1976836032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c4117800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b0a0 tx=0x7f88bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 -- 172.21.15.16:0/1976836032 shutdown_connections 2024-05-27T06:59:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 --2- 172.21.15.16:0/1976836032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c4117800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 --2- 172.21.15.16:0/1976836032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 0x7f88c4110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:33 smithi016 ceph-mon[31415]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 --2- 172.21.15.16:0/1976836032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 -- 172.21.15.16:0/1976836032 >> 172.21.15.16:0/1976836032 conn(0x7f88c4077c70 msgr2=0x7f88c40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.587+0000 7f88cae00700 1 -- 172.21.15.16:0/1976836032 shutdown_connections 2024-05-27T06:59:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.588+0000 7f88cae00700 1 -- 172.21.15.16:0/1976836032 wait complete. 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 Processor -- start 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 -- start start 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 0x7f88c41a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88c411a9b0 con 0x7f88c410b2d0 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88c411a830 con 0x7f88c410a980 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.589+0000 7f88cae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88c411ab30 con 0x7f88c4110960 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57888/0 (socket says 172.21.15.16:57888) 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 -- 172.21.15.16:0/3477409648 learned_addr learned my addr 172.21.15.16:0/3477409648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 0x7f88c41a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3e00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:33 smithi164 ceph-mon[35327]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 -- 172.21.15.16:0/3477409648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 msgr2=0x7f88c41a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 0x7f88c41a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 -- 172.21.15.16:0/3477409648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 msgr2=0x7f88c41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3400700 1 -- 172.21.15.16:0/3477409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c41a61f0 con 0x7f88c410b2d0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.590+0000 7f88c3e00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88c3400700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c970 tx=0x7f88b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88c0c00700 1 -- 172.21.15.16:0/3477409648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b000ce60 con 0x7f88c410b2d0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88c0c00700 1 -- 172.21.15.16:0/3477409648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88b0004830 con 0x7f88c410b2d0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c41a6480 con 0x7f88c410b2d0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88c0c00700 1 -- 172.21.15.16:0/3477409648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0005420 con 0x7f88c410b2d0 2024-05-27T06:59:33.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.591+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41a6970 con 0x7f88c410b2d0 2024-05-27T06:59:33.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.596+0000 7f88c0c00700 1 -- 172.21.15.16:0/3477409648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f88b000f450 con 0x7f88c410b2d0 2024-05-27T06:59:33.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.596+0000 7f88d9a21b80 1 Processor -- start 2024-05-27T06:59:33.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.596+0000 7f88d9a21b80 1 -- start start 2024-05-27T06:59:33.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb13ae20 0x55bfcb1470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb068df0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb058990 con 0x55bfcafbbdf0 2024-05-27T06:59:33.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb05c9d0 con 0x55bfcb13ae20 2024-05-27T06:59:33.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57902/0 (socket says 172.21.15.16:57902) 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.597+0000 7f88c8e00700 1 -- 172.21.15.16:0/2280178675 learned_addr learned my addr 172.21.15.16:0/2280178675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c3e00700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c3400700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb13ae20 0x55bfcb1470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c0c00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 0x7f88a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 -- 172.21.15.16:0/2280178675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb13ae20 msgr2=0x55bfcb1470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb13ae20 0x55bfcb1470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 -- 172.21.15.16:0/2280178675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 msgr2=0x55bfcb13a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 -- 172.21.15.16:0/2280178675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfcb0213d0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c3e00700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:33.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c0c00700 1 -- 172.21.15.16:0/3477409648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f88b00e2230 con 0x7f88c410b2d0 2024-05-27T06:59:33.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.598+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 0x7f88a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.599+0000 7f88c8e00700 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b0a0 tx=0x7f88bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6358fd33417e7cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.599+0000 7f88b9600700 1 -- 172.21.15.16:0/2280178675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc022030 con 0x55bfcb0a79a0 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.599+0000 7f88b9600700 1 -- 172.21.15.16:0/2280178675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88bc009500 con 0x55bfcb0a79a0 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.599+0000 7f88b9600700 1 -- 172.21.15.16:0/2280178675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0053d0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.599+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 0x7f88a80b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88bc02b000 tx=0x7f88bc02e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 -- 172.21.15.16:0/2280178675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 msgr2=0x55bfcb13a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b0a0 tx=0x7f88bc009260 comp rx=0 tx=0).stop 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 -- 172.21.15.16:0/2280178675 shutdown_connections 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb13ae20 0x55bfcb1470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13a2e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/2280178675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 -- 172.21.15.16:0/2280178675 >> 172.21.15.16:0/2280178675 conn(0x55bfcb055340 msgr2=0x55bfcb136470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 -- 172.21.15.16:0/2280178675 shutdown_connections 2024-05-27T06:59:33.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.600+0000 7f88d9a21b80 1 -- 172.21.15.16:0/2280178675 wait complete. 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.601+0000 7f88d9a21b80 1 -- start start 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.601+0000 7f88d9a21b80 1 Processor -- start 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb144620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb13ae20 0x55bfcb1450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfcaf5ef50 con 0x55bfcb0a79a0 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfcaff9710 con 0x55bfcb13ae20 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb147b30 con 0x55bfcafbbdf0 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88d9a21b80 10 client.0 _ll_register_callbacks cb 0x55bfcb002c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88c3400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88c3400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57910/0 (socket says 172.21.15.16:57910) 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88c3400700 1 -- 172.21.15.16:0/206978127 learned_addr learned my addr 172.21.15.16:0/206978127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.602+0000 7f88c3e00700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 -- 172.21.15.16:0/206978127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 msgr2=0x55bfcb144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 -- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb13ae20 msgr2=0x55bfcb1450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb13ae20 0x55bfcb1450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 -- 172.21.15.16:0/206978127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfcafa8880 con 0x55bfcb0a79a0 2024-05-27T06:59:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88c3400700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f88b00a93d0 tx=0x7f88b00a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c167dc060da9c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88ae000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b001d040 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f88b00a56d0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfcb06ad40 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88b00a9440 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88d9a21b80 10 client.15105 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.603+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfcb029010 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bfcb06ad40 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f88b00a96a0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 10 client.15105 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 20 client.15105 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb154b60 0x55bfcb155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 4 client.15105 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:59:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.604+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfcb064520 con 0x55bfcb154b60 2024-05-27T06:59:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.605+0000 7f88c3e00700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb154b60 0x55bfcb155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.606+0000 7f88c3e00700 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb154b60 0x55bfcb155fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8c65ed6ebe239df server_cookie=f9fe62e3f0271479 in_seq=0 out_seq=0 2024-05-27T06:59:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.606+0000 7f88ae000700 10 client.15105 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.608+0000 7f88ae000700 1 -- 172.21.15.16:0/206978127 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bfcb064520 con 0x55bfcb154b60 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.608+0000 7f88ae000700 10 client.15105 handle_command_reply: tid=0 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.608+0000 7f88d9a21b80 1 client.15105 shutdown 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.608+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 msgr2=0x55bfcb144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.608+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f88b00a93d0 tx=0x7f88b00a8c60 comp rx=0 tx=0).stop 2024-05-27T06:59:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 shutdown_connections 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb154b60 0x55bfcb155fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb13ae20 0x55bfcb1450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/206978127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb0a79a0 0x55bfcb144b60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 >> 172.21.15.16:0/206978127 conn(0x55bfcb055340 msgr2=0x55bfcb13a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88ae000700 0 client.15105 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 shutdown_connections 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 1 -- 172.21.15.16:0/206978127 wait complete. 2024-05-27T06:59:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.609+0000 7f88d9a21b80 20 client.15105 trim_cache size 0 max 16384 2024-05-27T06:59:33.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:33 smithi069 ceph-mon[35276]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:33.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.631+0000 7f88d9a21b80 1 Processor -- start 2024-05-27T06:59:33.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.631+0000 7f88d9a21b80 1 -- start start 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb136610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb154310 con 0x55bfcb13ae20 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb1549e0 con 0x55bfcafbbdf0 2024-05-27T06:59:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb1550b0 con 0x55bfcb0a79a0 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57926/0 (socket says 172.21.15.16:57926) 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88c8e00700 1 -- 172.21.15.16:0/3925490303 learned_addr learned my addr 172.21.15.16:0/3925490303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.632+0000 7f88c3e00700 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c3400700 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 -- 172.21.15.16:0/3925490303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb0a79a0 msgr2=0x55bfcb13fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 -- 172.21.15.16:0/3925490303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 msgr2=0x55bfcb136610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb136610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 -- 172.21.15.16:0/3925490303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfcb145410 con 0x55bfcb13ae20 2024-05-27T06:59:33.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88c8e00700 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0062a0 tx=0x7f88bc009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cca58233da0285 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88b8c00700 1 -- 172.21.15.16:0/3925490303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc037070 con 0x55bfcb13ae20 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.633+0000 7f88b8c00700 1 -- 172.21.15.16:0/3925490303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88bc005810 con 0x55bfcb13ae20 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88b8c00700 1 -- 172.21.15.16:0/3925490303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc022040 con 0x55bfcb13ae20 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 -- 172.21.15.16:0/3925490303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 msgr2=0x55bfcb13a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0062a0 tx=0x7f88bc009f40 comp rx=0 tx=0).stop 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 -- 172.21.15.16:0/3925490303 shutdown_connections 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13a0d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcafbbdf0 0x55bfcb136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/3925490303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 -- 172.21.15.16:0/3925490303 >> 172.21.15.16:0/3925490303 conn(0x55bfcb055340 msgr2=0x55bfcb135f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 -- 172.21.15.16:0/3925490303 shutdown_connections 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.634+0000 7f88d9a21b80 1 -- 172.21.15.16:0/3925490303 wait complete. 2024-05-27T06:59:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.635+0000 7f88d9a21b80 1 -- start start 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.635+0000 7f88d9a21b80 1 Processor -- start 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb155e50 con 0x55bfcb13ae20 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb155780 con 0x55bfcb0a79a0 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb156520 con 0x55bfcafbbdf0 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88d9a21b80 10 client.0 _ll_register_callbacks cb 0x55bfcb002c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57938/0 (socket says 172.21.15.16:57938) 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 -- 172.21.15.16:0/485461297 learned_addr learned my addr 172.21.15.16:0/485461297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c3e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c3400700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 -- 172.21.15.16:0/485461297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 msgr2=0x55bfcb13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 -- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb0a79a0 msgr2=0x55bfcb13f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.636+0000 7f88c8e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88c8e00700 1 -- 172.21.15.16:0/485461297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfcb15b710 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88c8e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005590 tx=0x7f88bc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb81d903a5be88b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88ae000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc040070 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc005810 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f88bc007850 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88bc037050 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88d9a21b80 10 client.15111 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.637+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfcb1c03b0 con 0x55bfcb13ae20 2024-05-27T06:59:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88bc005810 con 0x55bfcb13ae20 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f88bc011070 con 0x55bfcb13ae20 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 10 client.15111 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 20 client.15111 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb1c2130 0x55bfcb1c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 4 client.15111 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:59:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.638+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "status"}) -- 0x55bfcaedb200 con 0x55bfcb1c2130 2024-05-27T06:59:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.639+0000 7f88c3e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb1c2130 0x55bfcb1c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.640+0000 7f88c3e00700 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb1c2130 0x55bfcb1c35a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98e6df6dd714482e server_cookie=94e6bf609ea417ac in_seq=0 out_seq=0 2024-05-27T06:59:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.640+0000 7f88ae000700 10 client.15111 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T06:59:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88ae000700 1 -- 172.21.15.16:0/485461297 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bfcaedb200 con 0x55bfcb1c2130 2024-05-27T06:59:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88ae000700 10 client.15111 handle_command_reply: tid=0 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 client.15111 shutdown 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 msgr2=0x55bfcb13f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005590 tx=0x7f88bc00b880 comp rx=0 tx=0).stop 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 shutdown_connections 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bfcb1c2130 0x55bfcb1c35a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfcb13ae20 0x55bfcb13f9e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfcafbbdf0 0x55bfcb13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 --2- 172.21.15.16:0/485461297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfcb0a79a0 0x55bfcb13f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88ae000700 0 client.15111 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 >> 172.21.15.16:0/485461297 conn(0x55bfcb055340 msgr2=0x55bfcb14fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 shutdown_connections 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 1 -- 172.21.15.16:0/485461297 wait complete. 2024-05-27T06:59:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.641+0000 7f88d9a21b80 20 client.15111 trim_cache size 0 max 16384 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34478, 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 174.290946091, 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T06:59:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 303.22055191200002 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 msgr2=0x7f88a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 0x7f88a80b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88bc02b000 tx=0x7f88bc02e000 comp rx=0 tx=0).stop 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 msgr2=0x7f88c41a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c970 tx=0x7f88b000cc80 comp rx=0 tx=0).stop 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 shutdown_connections 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f88a80b2880 0x7f88a80b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c4110960 0x7f88c41a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88c410a980 0x7f88c41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 --2- 172.21.15.16:0/3477409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c410b2d0 0x7f88c41a18c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 >> 172.21.15.16:0/3477409648 conn(0x7f88c4077c70 msgr2=0x7f88c407b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.642+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 shutdown_connections 2024-05-27T06:59:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:33.643+0000 7f88cae00700 1 -- 172.21.15.16:0/3477409648 wait complete. 2024-05-27T06:59:34.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T06:59:35.052 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:35.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:35 smithi016 ceph-mon[31415]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:35.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:35 smithi164 ceph-mon[35327]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:35.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:35 smithi069 ceph-mon[35276]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:37.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:37 smithi164 ceph-mon[35327]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:37 smithi016 ceph-mon[31415]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:37.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:37 smithi069 ceph-mon[35276]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.649+0000 7fadd7e00700 1 -- 172.21.15.16:0/1010497853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0107e00 msgr2=0x7fadd0108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.649+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1010497853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0107e00 0x7fadd0108210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0099c0 tx=0x7fadbc009cd0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.649+0000 7fadd7e00700 1 -- 172.21.15.16:0/1010497853 shutdown_connections 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.649+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1010497853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 0x7fadd01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.650+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1010497853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd01034d0 0x7fadd01038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.650+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1010497853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0107e00 0x7fadd0108210 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.650+0000 7fadd7e00700 1 -- 172.21.15.16:0/1010497853 >> 172.21.15.16:0/1010497853 conn(0x7fadd00fef20 msgr2=0x7fadd0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.650+0000 7fadd7e00700 1 -- 172.21.15.16:0/1010497853 shutdown_connections 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.650+0000 7fadd7e00700 1 -- 172.21.15.16:0/1010497853 wait complete. 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.651+0000 7fadd7e00700 1 Processor -- start 2024-05-27T06:59:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.651+0000 7fadd7e00700 1 -- start start 2024-05-27T06:59:37.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 0x7fadd00ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 0x7fadd00ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0113260 con 0x7fadd01034d0 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01130e0 con 0x7fadd0108750 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd7e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01133e0 con 0x7fadd0107e00 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57962/0 (socket says 172.21.15.16:57962) 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd5400700 1 -- 172.21.15.16:0/1348113332 learned_addr learned my addr 172.21.15.16:0/1348113332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 0x7fadd00ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.652+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 0x7fadd00ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 -- 172.21.15.16:0/1348113332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 msgr2=0x7fadd00ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 0x7fadd00ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 -- 172.21.15.16:0/1348113332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 msgr2=0x7fadd00ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 0x7fadd00ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 -- 172.21.15.16:0/1348113332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01a9140 con 0x7fadd01034d0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 0x7fadd00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd5400700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d950 tx=0x7fadc000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadce000700 1 -- 172.21.15.16:0/1348113332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0016070 con 0x7fadd01034d0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01a9430 con 0x7fadd01034d0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadce000700 1 -- 172.21.15.16:0/1348113332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc0010460 con 0x7fadd01034d0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.653+0000 7fadce000700 1 -- 172.21.15.16:0/1348113332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000f620 con 0x7fadd01034d0 2024-05-27T06:59:37.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.654+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01a9990 con 0x7fadd01034d0 2024-05-27T06:59:37.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.658+0000 7fadce000700 1 -- 172.21.15.16:0/1348113332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadc0019070 con 0x7fadd01034d0 2024-05-27T06:59:37.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.658+0000 7fade689bb80 1 Processor -- start 2024-05-27T06:59:37.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 -- start start 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b72ba0df0 con 0x563b72c72d60 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b72b90990 con 0x563b72af3df0 2024-05-27T06:59:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.659+0000 7fade689bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b72b949d0 con 0x563b72c78ce0 2024-05-27T06:59:37.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57964/0 (socket says 172.21.15.16:57964) 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 -- 172.21.15.16:0/3473618189 learned_addr learned my addr 172.21.15.16:0/3473618189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadce000700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd5400700 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd5e00700 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadce000700 1 -- 172.21.15.16:0/1348113332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fadc00e2260 con 0x7fadd01034d0 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 -- 172.21.15.16:0/3473618189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 msgr2=0x563b72c72820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c72820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 -- 172.21.15.16:0/3473618189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 msgr2=0x563b72c787a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c787a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.660+0000 7fadd4a00700 1 -- 172.21.15.16:0/3473618189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b72b593d0 con 0x563b72c72d60 2024-05-27T06:59:37.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.661+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadd0076aa0 tx=0x7fadbc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bdd30481db99e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.661+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 0x7fadb40b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563b72c73ae0 tx=0x7fadc8009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.661+0000 7fadc6a00700 1 -- 172.21.15.16:0/3473618189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc026070 con 0x563b72c72d60 2024-05-27T06:59:37.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.661+0000 7fadc6a00700 1 -- 172.21.15.16:0/3473618189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadbc005650 con 0x563b72c72d60 2024-05-27T06:59:37.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.661+0000 7fadc6a00700 1 -- 172.21.15.16:0/3473618189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0038e0 con 0x563b72c72d60 2024-05-27T06:59:37.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.662+0000 7fade689bb80 1 -- 172.21.15.16:0/3473618189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 msgr2=0x563b72c73170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.662+0000 7fade689bb80 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadd0076aa0 tx=0x7fadbc0096c0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 -- 172.21.15.16:0/3473618189 shutdown_connections 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c73170 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c787a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 --2- 172.21.15.16:0/3473618189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 -- 172.21.15.16:0/3473618189 >> 172.21.15.16:0/3473618189 conn(0x563b72b53f70 msgr2=0x563b72c6d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 -- 172.21.15.16:0/3473618189 shutdown_connections 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 -- 172.21.15.16:0/3473618189 wait complete. 2024-05-27T06:59:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.663+0000 7fade689bb80 1 -- start start 2024-05-27T06:59:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.664+0000 7fade689bb80 1 Processor -- start 2024-05-27T06:59:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.664+0000 7fade689bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.664+0000 7fade689bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.664+0000 7fade689bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fade689bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b72a96f50 con 0x563b72c72d60 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fade689bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b72b31710 con 0x563b72af3df0 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fade689bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b72c7f750 con 0x563b72c78ce0 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fade689bb80 10 client.0 _ll_register_callbacks cb 0x563b72b3ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57980/0 (socket says 172.21.15.16:57980) 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 -- 172.21.15.16:0/3259349073 learned_addr learned my addr 172.21.15.16:0/3259349073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd5400700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd5e00700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 -- 172.21.15.16:0/3259349073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 msgr2=0x563b72c7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c7cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 -- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 msgr2=0x563b72c7c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c7c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.665+0000 7fadd4a00700 1 -- 172.21.15.16:0/3259349073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b72ae0880 con 0x563b72c72d60 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fadd4a00700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fadbc024040 tx=0x7fadbc00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91349a9d6455aa84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fadbb400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc02f060 con 0x563b72c72d60 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fadbc003680 con 0x563b72c72d60 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadbc005650 con 0x563b72c72d60 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadbc036050 con 0x563b72c72d60 2024-05-27T06:59:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.666+0000 7fade689bb80 10 client.15123 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b72b61010 con 0x563b72c72d60 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadbc005650 con 0x563b72c72d60 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fadbc00f8b0 con 0x563b72c72d60 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 10 client.15123 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 20 client.15123 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72c8b3a0 0x563b72c8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 4 client.15123 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T06:59:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.667+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b72ba2d40 con 0x563b72c8b3a0 2024-05-27T06:59:37.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.668+0000 7fadd5400700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72c8b3a0 0x563b72c8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.669+0000 7fadd5400700 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72c8b3a0 0x563b72c8d500 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8911d15fbd4cbc3f server_cookie=6f9bbe257553e211 in_seq=0 out_seq=0 2024-05-27T06:59:37.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.669+0000 7fadbb400700 10 client.15123 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T06:59:37.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fadbb400700 1 -- 172.21.15.16:0/3259349073 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b72ba2d40 con 0x563b72c8b3a0 2024-05-27T06:59:37.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fadbb400700 10 client.15123 handle_command_reply: tid=0 2024-05-27T06:59:37.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 client.15123 shutdown 2024-05-27T06:59:37.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 msgr2=0x563b72c7ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fadbc024040 tx=0x7fadbc00b9c0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 shutdown_connections 2024-05-27T06:59:37.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72c8b3a0 0x563b72c8d500 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fadbb400700 0 client.15123 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c7c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 --2- 172.21.15.16:0/3259349073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c7ca30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.671+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 >> 172.21.15.16:0/3259349073 conn(0x563b72b53f70 msgr2=0x563b72c6d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.672+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 shutdown_connections 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.672+0000 7fade689bb80 1 -- 172.21.15.16:0/3259349073 wait complete. 2024-05-27T06:59:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.672+0000 7fade689bb80 20 client.15123 trim_cache size 0 max 16384 2024-05-27T06:59:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.695+0000 7fade689bb80 1 Processor -- start 2024-05-27T06:59:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 -- start start 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8b840 con 0x563b72c72d60 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8bf10 con 0x563b72af3df0 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fade689bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8c5e0 con 0x563b72c78ce0 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fadd4a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57982/0 (socket says 172.21.15.16:57982) 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.696+0000 7fadd4a00700 1 -- 172.21.15.16:0/1968417753 learned_addr learned my addr 172.21.15.16:0/1968417753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd5400700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 -- 172.21.15.16:0/1968417753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 msgr2=0x563b72c889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 -- 172.21.15.16:0/1968417753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 msgr2=0x563b72c72820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c72820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 -- 172.21.15.16:0/1968417753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b72c7aa80 con 0x563b72c72d60 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd5400700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.697+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00fcb0 tx=0x7fadbc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1857304226e65558 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.698+0000 7fadc6000700 1 -- 172.21.15.16:0/1968417753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc02f040 con 0x563b72c72d60 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.698+0000 7fadc6000700 1 -- 172.21.15.16:0/1968417753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadbc0055d0 con 0x563b72c72d60 2024-05-27T06:59:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.698+0000 7fadc6000700 1 -- 172.21.15.16:0/1968417753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc026070 con 0x563b72c72d60 2024-05-27T06:59:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.698+0000 7fade689bb80 1 -- 172.21.15.16:0/1968417753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 msgr2=0x563b72c76bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.698+0000 7fade689bb80 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00fcb0 tx=0x7fadbc00bfc0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 -- 172.21.15.16:0/1968417753 shutdown_connections 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72c78ce0 0x563b72c889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72af3df0 0x563b72c72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 --2- 172.21.15.16:0/1968417753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c72d60 0x563b72c76bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 -- 172.21.15.16:0/1968417753 >> 172.21.15.16:0/1968417753 conn(0x563b72b53f70 msgr2=0x563b72c7c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 -- 172.21.15.16:0/1968417753 shutdown_connections 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 -- 172.21.15.16:0/1968417753 wait complete. 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 -- start start 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.699+0000 7fade689bb80 1 Processor -- start 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 0x563b72c970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 0x563b72c7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8d380 con 0x563b72c78ce0 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8ccb0 con 0x563b72c72d60 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b72c8da50 con 0x563b72af3df0 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fade689bb80 10 client.0 _ll_register_callbacks cb 0x563b72b3ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fadd5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fadd5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57984/0 (socket says 172.21.15.16:57984) 2024-05-27T06:59:37.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.700+0000 7fadd5e00700 1 -- 172.21.15.16:0/1437827140 learned_addr learned my addr 172.21.15.16:0/1437827140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5400700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 0x563b72c970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 0x563b72c7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 -- 172.21.15.16:0/1437827140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 msgr2=0x563b72c970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 0x563b72c970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 -- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 msgr2=0x563b72c7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 0x563b72c7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 -- 172.21.15.16:0/1437827140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b72c7e7c0 con 0x563b72c78ce0 2024-05-27T06:59:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd4a00700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 0x563b72c7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5400700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 0x563b72c970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadd5e00700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fadc8018d50 tx=0x7fadc8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35614600b4f17958 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.701+0000 7fadbb400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc802c060 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fadc80273f0 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadbc0055d0 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadc8004bb0 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fade689bb80 10 client.15129 fetch_fsmap learned FSMap version 30 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b72cf7820 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadbc0055d0 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.702+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fadc8005020 con 0x563b72c78ce0 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 10 client.15129 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T06:59:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T06:59:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 20 client.15129 populate_metadata read hostname 'smithi016' 2024-05-27T06:59:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72cf9d20 0x563b72cfb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 4 client.15129 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T06:59:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "status"}) -- 0x563b72b9c520 con 0x563b72cf9d20 2024-05-27T06:59:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.703+0000 7fadd5400700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72cf9d20 0x563b72cfb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.704+0000 7fadd5400700 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72cf9d20 0x563b72cfb190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e200e829cb3a89c server_cookie=81b405e29d0eeb8f in_seq=0 out_seq=0 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.704+0000 7fadbb400700 10 client.15129 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.705+0000 7fadbb400700 1 -- 172.21.15.16:0/1437827140 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563b72b9c520 con 0x563b72cf9d20 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.705+0000 7fadbb400700 10 client.15129 handle_command_reply: tid=0 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.705+0000 7fade689bb80 1 client.15129 shutdown 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 msgr2=0x563b72c7e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fadc8018d50 tx=0x7fadc8004b40 comp rx=0 tx=0).stop 2024-05-27T06:59:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 shutdown_connections 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563b72cf9d20 0x563b72cfb190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b72c78ce0 0x563b72c7e020 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b72af3df0 0x563b72c970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 --2- 172.21.15.16:0/1437827140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b72c72d60 0x563b72c7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fadbb400700 0 client.15129 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 >> 172.21.15.16:0/1437827140 conn(0x563b72b53f70 msgr2=0x563b72c76950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 shutdown_connections 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 1 -- 172.21.15.16:0/1437827140 wait complete. 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.706+0000 7fade689bb80 20 client.15129 trim_cache size 0 max 16384 2024-05-27T06:59:37.700 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34517, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 263.31110224999998, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 288.94806209799998 2024-05-27T06:59:37.701 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 msgr2=0x7fadb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 0x7fadb40b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563b72c73ae0 tx=0x7fadc8009680 comp rx=0 tx=0).stop 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 msgr2=0x7fadd00ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d950 tx=0x7fadc000dd10 comp rx=0 tx=0).stop 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 shutdown_connections 2024-05-27T06:59:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd0108750 0x7fadd00ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd01034d0 0x7fadd00ff400 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 --2- 172.21.15.16:0/1348113332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadd0107e00 0x7fadd00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.707+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 >> 172.21.15.16:0/1348113332 conn(0x7fadd00fef20 msgr2=0x7fadd006c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.708+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 shutdown_connections 2024-05-27T06:59:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:37.708+0000 7fadd7e00700 1 -- 172.21.15.16:0/1348113332 wait complete. 2024-05-27T06:59:38.696 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T06:59:38.697 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T06:59:38.707 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T06:59:38.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T06:59:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:39.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:39 smithi016 ceph-mon[31415]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:39.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:39 smithi164 ceph-mon[35327]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:39.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:39 smithi069 ceph-mon[35276]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:41.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:41 smithi016 ceph-mon[31415]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:41.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:41 smithi164 ceph-mon[35327]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:41.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:41 smithi069 ceph-mon[35276]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:41.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.759+0000 7fd044a00700 1 -- 172.21.15.16:0/2535763732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 msgr2=0x7fd040103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.759+0000 7fd044a00700 1 --2- 172.21.15.16:0/2535763732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd040103640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 -- 172.21.15.16:0/2535763732 shutdown_connections 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 --2- 172.21.15.16:0/2535763732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104d00 0x7fd04010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 --2- 172.21.15.16:0/2535763732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd040103640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 --2- 172.21.15.16:0/2535763732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104360 0x7fd040104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 -- 172.21.15.16:0/2535763732 >> 172.21.15.16:0/2535763732 conn(0x7fd0400fe870 msgr2=0x7fd040100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 -- 172.21.15.16:0/2535763732 shutdown_connections 2024-05-27T06:59:41.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.760+0000 7fd044a00700 1 -- 172.21.15.16:0/2535763732 wait complete. 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.761+0000 7fd044a00700 1 Processor -- start 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.761+0000 7fd044a00700 1 -- start start 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 0x7fd04019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 0x7fd0401a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010eb70 con 0x7fd040103230 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010e9f0 con 0x7fd040104d00 2024-05-27T06:59:41.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd044a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd04010ecf0 con 0x7fd040104360 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd03de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd03de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58008/0 (socket says 172.21.15.16:58008) 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd03de00700 1 -- 172.21.15.16:0/991115075 learned_addr learned my addr 172.21.15.16:0/991115075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd03d400700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 0x7fd04019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.762+0000 7fd03e800700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 0x7fd0401a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 -- 172.21.15.16:0/991115075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 msgr2=0x7fd04019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 0x7fd04019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 -- 172.21.15.16:0/991115075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 msgr2=0x7fd0401a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 0x7fd0401a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 -- 172.21.15.16:0/991115075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd040105580 con 0x7fd040103230 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03d400700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 0x7fd04019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03e800700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 0x7fd0401a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:41.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd03de00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd030006010 tx=0x7fd030009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:41.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.763+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030026070 con 0x7fd040103230 2024-05-27T06:59:41.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.764+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0300040a0 con 0x7fd040103230 2024-05-27T06:59:41.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.764+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd040105810 con 0x7fd040103230 2024-05-27T06:59:41.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.764+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000fca0 con 0x7fd040103230 2024-05-27T06:59:41.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.764+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd040105cf0 con 0x7fd040103230 2024-05-27T06:59:41.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.765+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0400512b0 con 0x7fd040103230 2024-05-27T06:59:41.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.768+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd030029070 con 0x7fd040103230 2024-05-27T06:59:41.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.769+0000 7fd036a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 0x7fd0240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:41.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.769+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd0300e3650 con 0x7fd040103230 2024-05-27T06:59:41.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.769+0000 7fd03d400700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 0x7fd0240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:41.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.770+0000 7fd03d400700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 0x7fd0240b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005950 tx=0x7fd02c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:41.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:41.772+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd030013070 con 0x7fd040103230 2024-05-27T06:59:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.032+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd040100a30 con 0x7fd0240b2880 2024-05-27T06:59:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.034+0000 7fd036a00700 1 -- 172.21.15.16:0/991115075 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd040100a30 con 0x7fd0240b2880 2024-05-27T06:59:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-05-27T06:59:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-05-27T06:59:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-05-27T06:59:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 msgr2=0x7fd0240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 0x7fd0240b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005950 tx=0x7fd02c0058e0 comp rx=0 tx=0).stop 2024-05-27T06:59:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 msgr2=0x7fd04019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd030006010 tx=0x7fd030009510 comp rx=0 tx=0).stop 2024-05-27T06:59:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 shutdown_connections 2024-05-27T06:59:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fd0240b2880 0x7fd0240b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd040104d00 0x7fd0401a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd040103230 0x7fd04019a8c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:42.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 --2- 172.21.15.16:0/991115075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd040104360 0x7fd04019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:42.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.037+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 >> 172.21.15.16:0/991115075 conn(0x7fd0400fe870 msgr2=0x7fd040100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:42.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.038+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 shutdown_connections 2024-05-27T06:59:42.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:42.038+0000 7fd044a00700 1 -- 172.21.15.16:0/991115075 wait complete. 2024-05-27T06:59:43.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T06:59:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:43.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:43 smithi016 ceph-mon[31415]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:43.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:43 smithi016 ceph-mon[31415]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:43.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:43 smithi069 ceph-mon[35276]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:43.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:43 smithi069 ceph-mon[35276]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:43.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:43 smithi164 ceph-mon[35327]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:43.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:43 smithi164 ceph-mon[35327]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:45.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:45 smithi016 ceph-mon[31415]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:45.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:45 smithi069 ceph-mon[35276]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:45.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:45 smithi164 ceph-mon[35327]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 -- 172.21.15.16:0/1642026027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54104160 msgr2=0x7fdf54104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 --2- 172.21.15.16:0/1642026027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54104160 0x7fdf54104570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009990 tx=0x7fdf44009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 -- 172.21.15.16:0/1642026027 shutdown_connections 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 --2- 172.21.15.16:0/1642026027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54105c30 0x7fdf5410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 --2- 172.21.15.16:0/1642026027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54104160 0x7fdf54104570 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 --2- 172.21.15.16:0/1642026027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 0x7fdf541056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.861+0000 7fdf53e00700 1 -- 172.21.15.16:0/1642026027 >> 172.21.15.16:0/1642026027 conn(0x7fdf540ff7a0 msgr2=0x7fdf54101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.862+0000 7fdf53e00700 1 -- 172.21.15.16:0/1642026027 shutdown_connections 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.862+0000 7fdf53e00700 1 -- 172.21.15.16:0/1642026027 wait complete. 2024-05-27T06:59:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.863+0000 7fdf53e00700 1 Processor -- start 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.863+0000 7fdf53e00700 1 -- start start 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.863+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54104160 0x7fdf5419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 0x7fdf5419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410fab0 con 0x7fdf54105c30 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410f930 con 0x7fdf54105290 2024-05-27T06:59:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410fc30 con 0x7fdf54104160 2024-05-27T06:59:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50908/0 (socket says 172.21.15.16:50908) 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf53400700 1 -- 172.21.15.16:0/3447765285 learned_addr learned my addr 172.21.15.16:0/3447765285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf52000700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 0x7fdf5419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.864+0000 7fdf52a00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54104160 0x7fdf5419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 -- 172.21.15.16:0/3447765285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54104160 msgr2=0x7fdf5419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54104160 0x7fdf5419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 -- 172.21.15.16:0/3447765285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 msgr2=0x7fdf5419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 0x7fdf5419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 -- 172.21.15.16:0/3447765285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf541a4400 con 0x7fdf54105c30 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53400700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00ada0 tx=0x7fdf4c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c017020 con 0x7fdf54105c30 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf4c00ce10 con 0x7fdf54105c30 2024-05-27T06:59:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf541a46f0 con 0x7fdf54105c30 2024-05-27T06:59:45.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.865+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c005440 con 0x7fdf54105c30 2024-05-27T06:59:45.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.866+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf541a4be0 con 0x7fdf54105c30 2024-05-27T06:59:45.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.867+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf540512b0 con 0x7fdf54105c30 2024-05-27T06:59:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.870+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdf4c004ad0 con 0x7fdf54105c30 2024-05-27T06:59:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.871+0000 7fdf58a00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 0x7fdf380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.871+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdf4c0e23b0 con 0x7fdf54105c30 2024-05-27T06:59:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.872+0000 7fdf52a00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 0x7fdf380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.872+0000 7fdf52a00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 0x7fdf380b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5c0 tx=0x7fdf4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:45.875+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf4c021020 con 0x7fdf54105c30 2024-05-27T06:59:46.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.124+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdf54077100 con 0x7fdf380b2880 2024-05-27T06:59:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.166+0000 7fdf58a00700 1 -- 172.21.15.16:0/3447765285 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fdf54077100 con 0x7fdf380b2880 2024-05-27T06:59:46.159 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T06:59:46.159 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (5m) 2m ago 5m 82.1M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 d5b3822c5947 2024-05-27T06:59:46.159 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi069 running (4m) 2m ago 4m 92.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 3cb4a848d344 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi164 running (5m) 2m ago 5m 83.2M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 4385b4e5d82d 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (5m) 2m ago 5m 80.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 3af778ea9a2a 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi069 running (5m) 2m ago 5m 83.9M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 cece82aadf2a 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi164 running (5m) 2m ago 5m 82.2M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 a66e6e055a65 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (5m) 2m ago 5m 82.1M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 68d541561a76 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi069 running (5m) 2m ago 5m 83.7M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 c69f7a53817c 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi164 running (5m) 2m ago 5m 83.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 4523bd94ae83 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (5m) 2m ago 5m 80.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 1fa8905d0eff 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi069 running (5m) 2m ago 5m 83.7M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 9fe235a28b52 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi164 running (5m) 2m ago 5m 82.3M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 f3738bac5aee 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (16m) 2m ago 16m 598M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 91ab9bd87972 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi069 *:8443,8765 running (13m) 2m ago 13m 534M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 c263c450001c 2024-05-27T06:59:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi164 *:8443,8765 running (13m) 2m ago 13m 534M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 1f7404f5138d 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (16m) 2m ago 16m 118M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 2f7380fe9933 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi069 running (13m) 2m ago 13m 108M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 d4af107a3aec 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi164 running (14m) 2m ago 14m 109M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 0fc103f84c79 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (13m) 2m ago 13m 119M 4273M 19.0.0-3872-g38f9d900 e36ea4fc8a58 f51711abef3f 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi069 running (12m) 2m ago 12m 121M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 91d6a8d5594d 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi164 running (11m) 2m ago 11m 113M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 36a8b71970a3 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (11m) 2m ago 11m 118M 4273M 19.0.0-3872-g38f9d900 e36ea4fc8a58 4c8228784b22 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi069 running (10m) 2m ago 10m 118M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 de40c2f66383 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi164 running (10m) 2m ago 10m 111M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 55ef163c7e9c 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (9m) 2m ago 9m 116M 4273M 19.0.0-3872-g38f9d900 e36ea4fc8a58 63e813b3d54a 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi069 running (8m) 2m ago 8m 117M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 ab51c93013e6 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi164 running (8m) 2m ago 8m 118M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 c1a4b7f7106a 2024-05-27T06:59:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (7m) 2m ago 7m 109M 4273M 19.0.0-3872-g38f9d900 e36ea4fc8a58 85fbc6061b84 2024-05-27T06:59:46.162 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi069 running (6m) 2m ago 6m 112M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 e7c00c289edd 2024-05-27T06:59:46.162 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi164 running (6m) 2m ago 6m 109M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 0c16bdb0026d 2024-05-27T06:59:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 msgr2=0x7fdf380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 0x7fdf380b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5c0 tx=0x7fdf4401a040 comp rx=0 tx=0).stop 2024-05-27T06:59:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 msgr2=0x7fdf541a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00ada0 tx=0x7fdf4c00c330 comp rx=0 tx=0).stop 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 shutdown_connections 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fdf380b2880 0x7fdf380b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54105c30 0x7fdf541a3e10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54104160 0x7fdf5419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 --2- 172.21.15.16:0/3447765285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf54105290 0x7fdf5419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 >> 172.21.15.16:0/3447765285 conn(0x7fdf540ff7a0 msgr2=0x7fdf540759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 shutdown_connections 2024-05-27T06:59:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:46.171+0000 7fdf53e00700 1 -- 172.21.15.16:0/3447765285 wait complete. 2024-05-27T06:59:47.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T06:59:47.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:47 smithi016 ceph-mon[31415]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:47.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:47 smithi016 ceph-mon[31415]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:47.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:47 smithi069 ceph-mon[35276]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:47.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:47 smithi069 ceph-mon[35276]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:47 smithi164 ceph-mon[35327]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:47 smithi164 ceph-mon[35327]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:49.463 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:49 smithi016 ceph-mon[31415]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:49.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:49 smithi069 ceph-mon[35276]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:49.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:49 smithi164 ceph-mon[35327]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 -- 172.21.15.16:0/187310246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 msgr2=0x7f47d4106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 --2- 172.21.15.16:0/187310246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d4106790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009990 tx=0x7f47c4009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 -- 172.21.15.16:0/187310246 shutdown_connections 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 --2- 172.21.15.16:0/187310246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 0x7f47d410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 --2- 172.21.15.16:0/187310246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d4106790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 --2- 172.21.15.16:0/187310246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 0x7f47d41078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 -- 172.21.15.16:0/187310246 >> 172.21.15.16:0/187310246 conn(0x7f47d4101a00 msgr2=0x7f47d4103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.142+0000 7f47d3200700 1 -- 172.21.15.16:0/187310246 shutdown_connections 2024-05-27T06:59:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.143+0000 7f47d3200700 1 -- 172.21.15.16:0/187310246 wait complete. 2024-05-27T06:59:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 Processor -- start 2024-05-27T06:59:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 -- start start 2024-05-27T06:59:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 0x7f47d419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 0x7f47d41a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4111cd0 con 0x7f47d4106380 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.144+0000 7f47d3200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4111b50 con 0x7f47d4107e50 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4111e50 con 0x7f47d41074b0 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50934/0 (socket says 172.21.15.16:50934) 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 -- 172.21.15.16:0/431185563 learned_addr learned my addr 172.21.15.16:0/431185563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47c9400700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 0x7f47d419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d2800700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 0x7f47d41a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 -- 172.21.15.16:0/431185563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 msgr2=0x7f47d419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 0x7f47d419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 -- 172.21.15.16:0/431185563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 msgr2=0x7f47d41a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 0x7f47d41a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.145+0000 7f47d1e00700 1 -- 172.21.15.16:0/431185563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d41a4410 con 0x7f47d4106380 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47d1e00700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003820 tx=0x7f47c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4026070 con 0x7f47d4106380 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47c4004020 con 0x7f47d4106380 2024-05-27T06:59:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d41a46a0 con 0x7f47d4106380 2024-05-27T06:59:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400fc60 con 0x7f47d4106380 2024-05-27T06:59:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.146+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41a4af0 con 0x7f47d4106380 2024-05-27T06:59:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.148+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d40512b0 con 0x7f47d4106380 2024-05-27T06:59:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.150+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f47c4029070 con 0x7f47d4106380 2024-05-27T06:59:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.152+0000 7f47cb400700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 0x7f47b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.152+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f47c40e3450 con 0x7f47d4106380 2024-05-27T06:59:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.152+0000 7f47c9400700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 0x7f47b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.153+0000 7f47c9400700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 0x7f47b80b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009550 tx=0x7f47bc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.155+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47c4030080 con 0x7f47d4106380 2024-05-27T06:59:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.417+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f47d4063bf0 con 0x7f47b80b2880 2024-05-27T06:59:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.432+0000 7f47cb400700 1 -- 172.21.15.16:0/431185563 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f47d4063bf0 con 0x7f47b80b2880 2024-05-27T06:59:50.424 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T06:59:50.424 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 2m ago 5m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-05-27T06:59:50.424 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 2m ago 13m smithi016=x;smithi069=y;smithi164=z;count:3 2024-05-27T06:59:50.424 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 2m ago 14m smithi016:172.21.15.16=a;smithi069:172.21.15.69=b;smithi164:172.21.15.164=c;count:3 2024-05-27T06:59:50.425 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 2m ago - 2024-05-27T06:59:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 msgr2=0x7f47b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 0x7f47b80b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009550 tx=0x7f47bc0091b0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 msgr2=0x7f47d419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003820 tx=0x7f47c40047b0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 shutdown_connections 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f47b80b2880 0x7f47b80b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47d4107e50 0x7f47d41a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4106380 0x7f47d419c8b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 --2- 172.21.15.16:0/431185563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47d41074b0 0x7f47d419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 >> 172.21.15.16:0/431185563 conn(0x7f47d4101a00 msgr2=0x7f47d410a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 shutdown_connections 2024-05-27T06:59:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:50.435+0000 7f47d3200700 1 -- 172.21.15.16:0/431185563 wait complete. 2024-05-27T06:59:51.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T06:59:51.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:51 smithi016 ceph-mon[31415]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:51.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:51 smithi016 ceph-mon[31415]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:51.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:51 smithi069 ceph-mon[35276]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:51.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:51 smithi069 ceph-mon[35276]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:51.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:51 smithi164 ceph-mon[35327]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T06:59:51.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:51 smithi164 ceph-mon[35327]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:53.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:53 smithi016 ceph-mon[31415]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:53.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:53 smithi069 ceph-mon[35276]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:53.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:53 smithi164 ceph-mon[35327]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 -- 172.21.15.16:0/1721234304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c0760e0 msgr2=0x7f197c076570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 --2- 172.21.15.16:0/1721234304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c0760e0 0x7f197c076570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1968009990 tx=0x7f1968009ca0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 -- 172.21.15.16:0/1721234304 shutdown_connections 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 --2- 172.21.15.16:0/1721234304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c10c1e0 0x7f197c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 --2- 172.21.15.16:0/1721234304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 0x7f197c075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 --2- 172.21.15.16:0/1721234304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c0760e0 0x7f197c076570 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 -- 172.21.15.16:0/1721234304 >> 172.21.15.16:0/1721234304 conn(0x7f197c0ff9b0 msgr2=0x7f197c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 -- 172.21.15.16:0/1721234304 shutdown_connections 2024-05-27T06:59:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.294+0000 7f1983600700 1 -- 172.21.15.16:0/1721234304 wait complete. 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 Processor -- start 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 -- start start 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 0x7f197c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 0x7f197c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111e40 con 0x7f197c10c1e0 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111cc0 con 0x7f197c0776d0 2024-05-27T06:59:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.296+0000 7f1983600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111fc0 con 0x7f197c0760e0 2024-05-27T06:59:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47344/0 (socket says 172.21.15.16:47344) 2024-05-27T06:59:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 -- 172.21.15.16:0/1149773966 learned_addr learned my addr 172.21.15.16:0/1149773966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f197be00700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 0x7f197c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1980c00700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 0x7f197c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 -- 172.21.15.16:0/1149773966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 msgr2=0x7f197c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 0x7f197c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 -- 172.21.15.16:0/1149773966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 msgr2=0x7f197c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 0x7f197c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1981600700 1 -- 172.21.15.16:0/1149773966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c1a45a0 con 0x7f197c10c1e0 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.297+0000 7f1980c00700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 0x7f197c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1981600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1974007ee0 tx=0x7f197400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197400dbd0 con 0x7f197c10c1e0 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f197400dd70 con 0x7f197c10c1e0 2024-05-27T06:59:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740058d0 con 0x7f197c10c1e0 2024-05-27T06:59:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c1a4890 con 0x7f197c10c1e0 2024-05-27T06:59:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.298+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1a4dd0 con 0x7f197c10c1e0 2024-05-27T06:59:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.299+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c0512b0 con 0x7f197c10c1e0 2024-05-27T06:59:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.303+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f197401a020 con 0x7f197c10c1e0 2024-05-27T06:59:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.305+0000 7f1979600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 0x7f19640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.305+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f19740e2490 con 0x7f197c10c1e0 2024-05-27T06:59:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.305+0000 7f1980c00700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 0x7f19640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.306+0000 7f1980c00700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 0x7f19640b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1970005950 tx=0x7f19700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.308+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1974010070 con 0x7f197c10c1e0 2024-05-27T06:59:54.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.555+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f197c063bf0 con 0x7f19640b2900 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.558+0000 7f1979600700 1 -- 172.21.15.16:0/1149773966 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f197c063bf0 con 0x7f19640b2900 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stdout:smithi069 172.21.15.69 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stdout:smithi164 172.21.15.164 2024-05-27T06:59:54.551 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 msgr2=0x7f19640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 0x7f19640b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1970005950 tx=0x7f19700058e0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 msgr2=0x7f197c1a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1974007ee0 tx=0x7f197400d3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 shutdown_connections 2024-05-27T06:59:54.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f19640b2900 0x7f19640b4e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.562+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c10c1e0 0x7f197c1a3f20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.563+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f197c0760e0 0x7f197c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.563+0000 7f1983600700 1 --2- 172.21.15.16:0/1149773966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f197c0776d0 0x7f197c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.563+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 >> 172.21.15.16:0/1149773966 conn(0x7f197c0ff9b0 msgr2=0x7f197c108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.563+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 shutdown_connections 2024-05-27T06:59:54.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:54.563+0000 7f1983600700 1 -- 172.21.15.16:0/1149773966 wait complete. 2024-05-27T06:59:55.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T06:59:55.634 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:55 smithi164 ceph-mon[35327]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:55.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:55 smithi016 ceph-mon[31415]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:55.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:55 smithi069 ceph-mon[35276]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T06:59:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T06:59:56.575 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:56 smithi016 ceph-mon[31415]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:56.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:56 smithi164 ceph-mon[35327]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:56.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:56 smithi069 ceph-mon[35276]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T06:59:57.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:57 smithi164 ceph-mon[35327]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:57.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:57 smithi016 ceph-mon[31415]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:57.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:57 smithi069 ceph-mon[35276]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T06:59:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 -- 172.21.15.16:0/2057211052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c0776d0 msgr2=0x7efe1c075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 --2- 172.21.15.16:0/2057211052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c0776d0 0x7efe1c075ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe100099c0 tx=0x7efe10009cd0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 -- 172.21.15.16:0/2057211052 shutdown_connections 2024-05-27T06:59:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 --2- 172.21.15.16:0/2057211052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c110980 0x7efe1c112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 --2- 172.21.15.16:0/2057211052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c0776d0 0x7efe1c075ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 --2- 172.21.15.16:0/2057211052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 0x7efe1c076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 -- 172.21.15.16:0/2057211052 >> 172.21.15.16:0/2057211052 conn(0x7efe1c0ff9b0 msgr2=0x7efe1c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 -- 172.21.15.16:0/2057211052 shutdown_connections 2024-05-27T06:59:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.588+0000 7efe24600700 1 -- 172.21.15.16:0/2057211052 wait complete. 2024-05-27T06:59:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.589+0000 7efe24600700 1 Processor -- start 2024-05-27T06:59:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 -- start start 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 0x7efe1c1a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c0776d0 0x7efe1c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c116270 con 0x7efe1c110980 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c1160f0 con 0x7efe1c0776d0 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe24600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c1163f0 con 0x7efe1c0760e0 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.590+0000 7efe22600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47352/0 (socket says 172.21.15.16:47352) 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 -- 172.21.15.16:0/3130998330 learned_addr learned my addr 172.21.15.16:0/3130998330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe21c00700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 0x7efe1c1a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe21200700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c0776d0 0x7efe1c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 -- 172.21.15.16:0/3130998330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 msgr2=0x7efe1c1a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 0x7efe1c1a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 -- 172.21.15.16:0/3130998330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c0776d0 msgr2=0x7efe1c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c0776d0 0x7efe1c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 -- 172.21.15.16:0/3130998330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c1a8a30 con 0x7efe1c110980 2024-05-27T06:59:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.591+0000 7efe22600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe14007ee0 tx=0x7efe1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.592+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1400ddd0 con 0x7efe1c110980 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.592+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe14004020 con 0x7efe1c110980 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.592+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14015660 con 0x7efe1c110980 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.592+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1a8d20 con 0x7efe1c110980 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.592+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1a91b0 con 0x7efe1c110980 2024-05-27T06:59:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.593+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c0512b0 con 0x7efe1c110980 2024-05-27T06:59:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.596+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efe140041d0 con 0x7efe1c110980 2024-05-27T06:59:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.597+0000 7efe1a000700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 0x7efe040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T06:59:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.597+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efe140e20f0 con 0x7efe1c110980 2024-05-27T06:59:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.598+0000 7efe21c00700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 0x7efe040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T06:59:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.598+0000 7efe21c00700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 0x7efe040b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe100098e0 tx=0x7efe10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T06:59:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.604+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe14010070 con 0x7efe1c110980 2024-05-27T06:59:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.852+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7efe1c063bf0 con 0x7efe040b2880 2024-05-27T06:59:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.856+0000 7efe1a000700 1 -- 172.21.15.16:0/3130998330 <== mgr.14152 v2:172.21.15.16:6800/2755801148 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7efe1c063bf0 con 0x7efe040b2880 2024-05-27T06:59:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T06:59:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 7m ago Has a FileSystem, LVM detected 2024-05-27T06:59:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-27T06:59:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-27T06:59:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 msgr2=0x7efe040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 0x7efe040b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe100098e0 tx=0x7efe10005fb0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 msgr2=0x7efe1c1a83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe14007ee0 tx=0x7efe1400d3b0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 shutdown_connections 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7efe040b2880 0x7efe040b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe1c110980 0x7efe1c1a83b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe1c0760e0 0x7efe1c1a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 --2- 172.21.15.16:0/3130998330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c0776d0 0x7efe1c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T06:59:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 >> 172.21.15.16:0/3130998330 conn(0x7efe1c0ff9b0 msgr2=0x7efe1c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T06:59:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 shutdown_connections 2024-05-27T06:59:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T06:59:58.859+0000 7efe24600700 1 -- 172.21.15.16:0/3130998330 wait complete. 2024-05-27T06:59:59.671 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 06:59:59 smithi069 ceph-mon[35276]: pgmap v540: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:59.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 06:59:59 smithi016 ceph-mon[31415]: pgmap v540: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T06:59:59.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 06:59:59 smithi164 ceph-mon[35327]: pgmap v540: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:00.076 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T07:00:00.085 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:00:00.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T07:00:00.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:00.706 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:00 smithi016 ceph-mon[31415]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:00.706 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:00 smithi016 ceph-mon[31415]: overall HEALTH_OK 2024-05-27T07:00:00.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:00 smithi164 ceph-mon[35327]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:00.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:00 smithi164 ceph-mon[35327]: overall HEALTH_OK 2024-05-27T07:00:00.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:00 smithi069 ceph-mon[35276]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:00.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:00 smithi069 ceph-mon[35276]: overall HEALTH_OK 2024-05-27T07:00:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:01 smithi016 ceph-mon[31415]: pgmap v541: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:01.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:01 smithi164 ceph-mon[35327]: pgmap v541: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:01.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:01 smithi069 ceph-mon[35276]: pgmap v541: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 -- 172.21.15.16:0/1451948135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 msgr2=0x7f91f0107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 --2- 172.21.15.16:0/1451948135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f0107260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009990 tx=0x7f91e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 -- 172.21.15.16:0/1451948135 shutdown_connections 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 --2- 172.21.15.16:0/1451948135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f010a100 0x7f91f010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 --2- 172.21.15.16:0/1451948135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f0107260 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 --2- 172.21.15.16:0/1451948135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f01077a0 0x7f91f0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 -- 172.21.15.16:0/1451948135 >> 172.21.15.16:0/1451948135 conn(0x7f91f00fe880 msgr2=0x7f91f0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 -- 172.21.15.16:0/1451948135 shutdown_connections 2024-05-27T07:00:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.153+0000 7f91f7800700 1 -- 172.21.15.16:0/1451948135 wait complete. 2024-05-27T07:00:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.154+0000 7f91f7800700 1 Processor -- start 2024-05-27T07:00:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 -- start start 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 0x7f91f019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f010a100 0x7f91f01a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010fc30 con 0x7f91f0104e40 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010fab0 con 0x7f91f01077a0 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f7800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010fdb0 con 0x7f91f010a100 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.155+0000 7f91f4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47270/0 (socket says 172.21.15.16:47270) 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 -- 172.21.15.16:0/2245181403 learned_addr learned my addr 172.21.15.16:0/2245181403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91efe00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 0x7f91f019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f5800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f010a100 0x7f91f01a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 -- 172.21.15.16:0/2245181403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f010a100 msgr2=0x7f91f01a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f010a100 0x7f91f01a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 -- 172.21.15.16:0/2245181403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 msgr2=0x7f91f019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 0x7f91f019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 -- 172.21.15.16:0/2245181403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f01a45a0 con 0x7f91f0104e40 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91efe00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 0x7f91f019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.156+0000 7f91f4e00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.157+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4027070 con 0x7f91f0104e40 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.157+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01a4830 con 0x7f91f0104e40 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.157+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91e40040a0 con 0x7f91f0104e40 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.157+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4022d00 con 0x7f91f0104e40 2024-05-27T07:00:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.157+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01a4c80 con 0x7f91f0104e40 2024-05-27T07:00:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.158+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f00512b0 con 0x7f91f0104e40 2024-05-27T07:00:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.161+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f91e402a070 con 0x7f91f0104e40 2024-05-27T07:00:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.163+0000 7f91ed600700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 0x7f91d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.164+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f91e40e3ea0 con 0x7f91f0104e40 2024-05-27T07:00:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.164+0000 7f91efe00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 0x7f91d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.165+0000 7f91efe00700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 0x7f91d80b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f91f019daf0 tx=0x7f91dc00b690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.166+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91e4013070 con 0x7f91f0104e40 2024-05-27T07:00:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.466+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f91f01a5160 con 0x7f91f0104e40 2024-05-27T07:00:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.467+0000 7f91ed600700 1 -- 172.21.15.16:0/2245181403 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+3088 (secure 0 0 0) 0x7f91e40ab840 con 0x7f91f0104e40 2024-05-27T07:00:03.460 INFO:teuthology.orchestra.run.smithi016.stdout:e30 2024-05-27T07:00:03.460 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:00:03.460 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 30 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-05-27T06:55:13.518131+0000 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-05-27T06:56:48.860392+0000 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-05-27T07:00:03.461 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34517,1=34478,2=34451,3=34421,4=24481} 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3,4] 2024-05-27T07:00:03.462 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 34517 members: 34517,34478,34451,24481,34421 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:34517} state up:active seq 8 addr [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.164:6836/162118671,v1:172.21.15.164:6837/162118671] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{1:34478} state up:active seq 34 addr [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/2731773245,v1:172.21.15.164:6835/2731773245] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{2:34451} state up:active seq 37 addr [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{2:24412} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/1294360695,v1:172.21.15.164:6833/1294360695] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{3:34421} state up:active seq 41 addr [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14595} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/1178863608,v1:172.21.15.16:6841/1178863608] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{4:24481} state up:active seq 33 addr [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.463 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14583} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/1261534836,v1:172.21.15.16:6839/1261534836] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.16:6834/553407450,v1:172.21.15.16:6835/553407450] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14523} state up:standby seq 1 addr [v2:172.21.15.16:6836/3811821671,v1:172.21.15.16:6837/3811821671] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:00:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.470+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 msgr2=0x7f91d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.470+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 0x7f91d80b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f91f019daf0 tx=0x7f91dc00b690 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.470+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 msgr2=0x7f91f019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.470+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4009510 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 shutdown_connections 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f91d80b2880 0x7f91d80b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f010a100 0x7f91f01a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f0104e40 0x7f91f019ca40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 --2- 172.21.15.16:0/2245181403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91f01077a0 0x7f91f019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 >> 172.21.15.16:0/2245181403 conn(0x7f91f00fe880 msgr2=0x7f91f0102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 shutdown_connections 2024-05-27T07:00:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:03.471+0000 7f91f7800700 1 -- 172.21.15.16:0/2245181403 wait complete. 2024-05-27T07:00:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T07:00:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:03 smithi164 ceph-mon[35327]: pgmap v542: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:03.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:03 smithi016 ceph-mon[31415]: pgmap v542: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:03.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:03 smithi069 ceph-mon[35276]: pgmap v542: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:04.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-27T07:00:04.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:04 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2245181403' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:00:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:04 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2245181403' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:00:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:04 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2245181403' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:00:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:05.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:05 smithi016 ceph-mon[31415]: pgmap v543: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:05.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:05 smithi164 ceph-mon[35327]: pgmap v543: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:05.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:05 smithi069 ceph-mon[35276]: pgmap v543: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:07.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 -- 172.21.15.16:0/2634122117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f04107470 msgr2=0x7f1f04109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:07.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2634122117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f04107470 0x7f1f04109950 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009990 tx=0x7f1eec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 -- 172.21.15.16:0/2634122117 shutdown_connections 2024-05-27T07:00:07.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2634122117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f0410c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2634122117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f0406c2c0 0x7f1f04106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2634122117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f04107470 0x7f1f04109950 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 -- 172.21.15.16:0/2634122117 >> 172.21.15.16:0/2634122117 conn(0x7f1f040fe870 msgr2=0x7f1f04100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 -- 172.21.15.16:0/2634122117 shutdown_connections 2024-05-27T07:00:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.459+0000 7f1f0b000700 1 -- 172.21.15.16:0/2634122117 wait complete. 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.460+0000 7f1f0b000700 1 Processor -- start 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.460+0000 7f1f0b000700 1 -- start start 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f04107470 0x7f1f0419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f041a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410fcd0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410fb50 con 0x7f1f04109fe0 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f0b000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410fe50 con 0x7f1f04107470 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f03e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f03e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47284/0 (socket says 172.21.15.16:47284) 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1f03e00700 1 -- 172.21.15.16:0/2900574976 learned_addr learned my addr 172.21.15.16:0/2900574976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.461+0000 7f1efbe00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f04107470 0x7f1f0419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f09000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f041a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 -- 172.21.15.16:0/2900574976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f04107470 msgr2=0x7f1f0419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f04107470 0x7f1f0419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 -- 172.21.15.16:0/2900574976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 msgr2=0x7f1f041a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f041a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 -- 172.21.15.16:0/2900574976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f041a23a0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f09000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f041a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.462+0000 7f1f03e00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400b530 tx=0x7f1ef400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.463+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4015020 con 0x7f1f0406c2c0 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.463+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ef4011cc0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.463+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4010dc0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.463+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f041a2690 con 0x7f1f0406c2c0 2024-05-27T07:00:07.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.463+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a2ca0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.464+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f040512b0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.467+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ef4018020 con 0x7f1f0406c2c0 2024-05-27T07:00:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.468+0000 7f1f01600700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 0x7f1ee80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.469+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1ef40e23b0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.469+0000 7f1efbe00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 0x7f1ee80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.470+0000 7f1efbe00700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 0x7f1ee80b4e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f0419b980 tx=0x7f1eec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.472+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ef401f020 con 0x7f1f0406c2c0 2024-05-27T07:00:07.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:07 smithi016 ceph-mon[31415]: pgmap v544: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.712+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1f0402e5e0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.714+0000 7f1f01600700 1 -- 172.21.15.16:0/2900574976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5672 (secure 0 0 0) 0x7f1ef40a9cc0 con 0x7f1f0406c2c0 2024-05-27T07:00:07.707 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 90 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-05-27T06:43:14.821531+0000 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-05-27T06:56:40.328992+0000 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 27 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-05-27T07:00:07.708 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/4171777191,v1:172.21.15.16:6803/4171777191] [v2:172.21.15.16:6804/4171777191,v1:172.21.15.16:6805/4171777191] exists,up ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6800/1028348959,v1:172.21.15.69:6801/1028348959] [v2:172.21.15.69:6802/1028348959,v1:172.21.15.69:6803/1028348959] exists,up 6f0fa1e6-b4a7-48a6-b89d-c5612a23e676 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/3071758957,v1:172.21.15.164:6801/3071758957] [v2:172.21.15.164:6802/3071758957,v1:172.21.15.164:6803/3071758957] exists,up ebccb12e-a7d2-4907-9140-78f8ab1994bc 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 25 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/3551792310,v1:172.21.15.16:6811/3551792310] [v2:172.21.15.16:6812/3551792310,v1:172.21.15.16:6813/3551792310] exists,up 0b1c5db9-adc3-444b-9eca-3599f6e32f89 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 31 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6808/823147772,v1:172.21.15.69:6809/823147772] [v2:172.21.15.69:6810/823147772,v1:172.21.15.69:6811/823147772] exists,up c25a93d3-53d0-4155-8981-50f926759f79 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 36 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/3562698825,v1:172.21.15.164:6809/3562698825] [v2:172.21.15.164:6810/3562698825,v1:172.21.15.164:6811/3562698825] exists,up e85ea8a0-f98e-416d-9d63-ed816d3afb04 2024-05-27T07:00:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 42 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/1434782901,v1:172.21.15.16:6819/1434782901] [v2:172.21.15.16:6820/1434782901,v1:172.21.15.16:6821/1434782901] exists,up be746ea8-a314-4561-9bf5-bc7d7944ffce 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6816/125788904,v1:172.21.15.69:6817/125788904] [v2:172.21.15.69:6818/125788904,v1:172.21.15.69:6819/125788904] exists,up a4dced21-90cd-4a3d-8823-02824ca1c42a 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 54 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/4237870285,v1:172.21.15.164:6817/4237870285] [v2:172.21.15.164:6818/4237870285,v1:172.21.15.164:6819/4237870285] exists,up d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/1571284516,v1:172.21.15.16:6827/1571284516] [v2:172.21.15.16:6828/1571284516,v1:172.21.15.16:6829/1571284516] exists,up 6e4f0c06-7a28-44a5-8ee0-84370c455177 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6824/1283060257,v1:172.21.15.69:6825/1283060257] [v2:172.21.15.69:6826/1283060257,v1:172.21.15.69:6827/1283060257] exists,up c705b502-5d5e-4b43-89db-40a14acec587 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/3936768359,v1:172.21.15.164:6825/3936768359] [v2:172.21.15.164:6826/3936768359,v1:172.21.15.164:6827/3936768359] exists,up e1b29f9e-85de-403e-bc32-f10ad4d089cd 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.6 [8,11] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.8 [2,11] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.14 [2,4] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1b [5,9] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.23 [4,8] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.28 [5,6] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.2d [3,9] 2024-05-27T07:00:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.36 [4,9] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.3a [4,7] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.13 [9,8] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.18 [9,8] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.19 [9,10] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.1a [5,8] 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3562581452 expires 2024-05-28T06:44:37.876629+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1801942445 expires 2024-05-28T06:44:37.876629+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2589421123 expires 2024-05-28T06:44:37.876629+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2390178341 expires 2024-05-28T06:44:37.876629+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3708593763 expires 2024-05-28T06:44:00.893187+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3327639895 expires 2024-05-28T06:44:00.893187+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/4243270602 expires 2024-05-28T06:44:00.893187+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/4120962675 expires 2024-05-28T06:44:00.893187+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/4120962675 expires 2024-05-28T06:44:00.893187+0000 2024-05-27T07:00:07.711 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2390178341 expires 2024-05-28T06:44:37.876629+0000 2024-05-27T07:00:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 msgr2=0x7f1ee80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 0x7f1ee80b4e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f0419b980 tx=0x7f1eec01a040 comp rx=0 tx=0).stop 2024-05-27T07:00:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 msgr2=0x7f1f0419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400b530 tx=0x7f1ef400b840 comp rx=0 tx=0).stop 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 shutdown_connections 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1ee80b2900 0x7f1ee80b4e10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f04109fe0 0x7f1f041a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0406c2c0 0x7f1f0419a8d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 --2- 172.21.15.16:0/2900574976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f04107470 0x7f1f0419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.717+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 >> 172.21.15.16:0/2900574976 conn(0x7f1f040fe870 msgr2=0x7f1f04105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.718+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 shutdown_connections 2024-05-27T07:00:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:07.718+0000 7f1f0b000700 1 -- 172.21.15.16:0/2900574976 wait complete. 2024-05-27T07:00:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:07 smithi164 ceph-mon[35327]: pgmap v544: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:07.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:07 smithi069 ceph-mon[35276]: pgmap v544: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:08.631 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-27T07:00:08.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:00:08.730 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:08 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2900574976' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:00:08.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:08 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2900574976' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:00:08.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:08 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2900574976' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:00:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:09.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:09 smithi016 ceph-mon[31415]: pgmap v545: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:09.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:09 smithi164 ceph-mon[35327]: pgmap v545: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:09.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:09 smithi069 ceph-mon[35276]: pgmap v545: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 -- 172.21.15.16:0/2124870713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c1151e0 msgr2=0x7f338c117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 --2- 172.21.15.16:0/2124870713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c1151e0 0x7f338c117600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f338400b0a0 tx=0x7f338400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 -- 172.21.15.16:0/2124870713 shutdown_connections 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 --2- 172.21.15.16:0/2124870713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c1151e0 0x7f338c117600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 --2- 172.21.15.16:0/2124870713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c077ce0 0x7f338c0780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 --2- 172.21.15.16:0/2124870713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c078630 0x7f338c114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 -- 172.21.15.16:0/2124870713 >> 172.21.15.16:0/2124870713 conn(0x7f338c102080 msgr2=0x7f338c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.657+0000 7f3394000700 1 -- 172.21.15.16:0/2124870713 shutdown_connections 2024-05-27T07:00:11.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:11 smithi016 ceph-mon[31415]: pgmap v546: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.658+0000 7f3394000700 1 -- 172.21.15.16:0/2124870713 wait complete. 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 Processor -- start 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 -- start start 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 0x7f338c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 0x7f338c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f338c11a970 con 0x7f338c077ce0 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f338c11a7f0 con 0x7f338c078630 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.659+0000 7f3394000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f338c11aaf0 con 0x7f338c1151e0 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47296/0 (socket says 172.21.15.16:47296) 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 -- 172.21.15.16:0/3503679112 learned_addr learned my addr 172.21.15.16:0/3503679112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3392000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 0x7f338c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3390c00700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 0x7f338c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 -- 172.21.15.16:0/3503679112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 msgr2=0x7f338c1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 0x7f338c1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 -- 172.21.15.16:0/3503679112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 msgr2=0x7f338c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 0x7f338c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3391600700 1 -- 172.21.15.16:0/3503679112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c1a61d0 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3392000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 0x7f338c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.660+0000 7f3390c00700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 0x7f338c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f3391600700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f338000d950 tx=0x7f338000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380016070 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3380010460 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c1a64c0 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000f620 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.661+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c1adde0 con 0x7f338c077ce0 2024-05-27T07:00:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.662+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338c0512b0 con 0x7f338c077ce0 2024-05-27T07:00:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.665+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3380019070 con 0x7f338c077ce0 2024-05-27T07:00:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.667+0000 7f338a000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.667+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f33800e2230 con 0x7f338c077ce0 2024-05-27T07:00:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.667+0000 7f3390c00700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.668+0000 7f3390c00700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 0x7f33740b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3378005e10 tx=0x7f3378005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.670+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33800a9b40 con 0x7f338c077ce0 2024-05-27T07:00:11.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:11 smithi164 ceph-mon[35327]: pgmap v546: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:11.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:11 smithi069 ceph-mon[35276]: pgmap v546: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:11.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.955+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f338c02e5e0 con 0x7f338c077ce0 2024-05-27T07:00:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.956+0000 7f338a000700 1 -- 172.21.15.16:0/3503679112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7f33800a9440 con 0x7f338c077ce0 2024-05-27T07:00:11.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:11.949 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 msgr2=0x7f33740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 0x7f33740b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3378005e10 tx=0x7f3378005d60 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 msgr2=0x7f338c1a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f338000d950 tx=0x7f338000dd10 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 shutdown_connections 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c1151e0 0x7f338c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c077ce0 0x7f338c1a1300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 --2- 172.21.15.16:0/3503679112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f338c078630 0x7f338c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.959+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 >> 172.21.15.16:0/3503679112 conn(0x7f338c102080 msgr2=0x7f338c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.960+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 shutdown_connections 2024-05-27T07:00:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:11.960+0000 7f3394000700 1 -- 172.21.15.16:0/3503679112 wait complete. 2024-05-27T07:00:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T07:00:12.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:12 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3503679112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:12.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:12 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3503679112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:12.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:12 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3503679112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:12.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:00:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:13.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:13 smithi016 ceph-mon[31415]: pgmap v547: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:13 smithi164 ceph-mon[35327]: pgmap v547: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:13.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:13 smithi069 ceph-mon[35276]: pgmap v547: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:14 smithi164 ceph-mon[35327]: pgmap v548: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:14.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:14 smithi016 ceph-mon[31415]: pgmap v548: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:14.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:14 smithi069 ceph-mon[35276]: pgmap v548: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:16.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 -- 172.21.15.16:0/1630192992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 msgr2=0x7f15d410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:16.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 --2- 172.21.15.16:0/1630192992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d410ad90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009990 tx=0x7f15c0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 -- 172.21.15.16:0/1630192992 shutdown_connections 2024-05-27T07:00:16.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 --2- 172.21.15.16:0/1630192992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d4117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 --2- 172.21.15.16:0/1630192992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 0x7f15d4110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 --2- 172.21.15.16:0/1630192992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d410ad90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 -- 172.21.15.16:0/1630192992 >> 172.21.15.16:0/1630192992 conn(0x7f15d4077c70 msgr2=0x7f15d40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:16.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 -- 172.21.15.16:0/1630192992 shutdown_connections 2024-05-27T07:00:16.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.080+0000 7f15dc600700 1 -- 172.21.15.16:0/1630192992 wait complete. 2024-05-27T07:00:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.081+0000 7f15dc600700 1 Processor -- start 2024-05-27T07:00:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 -- start start 2024-05-27T07:00:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 0x7f15d41a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15d411a950 con 0x7f15d410a980 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15d411a7d0 con 0x7f15d410b2d0 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15dc600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15d411aad0 con 0x7f15d4110960 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.082+0000 7f15d9c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48766/0 (socket says 172.21.15.16:48766) 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 -- 172.21.15.16:0/719133312 learned_addr learned my addr 172.21.15.16:0/719133312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9200700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15da600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 0x7f15d41a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 -- 172.21.15.16:0/719133312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 msgr2=0x7f15d41a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 0x7f15d41a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 -- 172.21.15.16:0/719133312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 msgr2=0x7f15d41a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d41a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 -- 172.21.15.16:0/719133312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41a6250 con 0x7f15d410a980 2024-05-27T07:00:16.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9200700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.083+0000 7f15d9c00700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f15c800d950 tx=0x7f15c800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.084+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8016070 con 0x7f15d410a980 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.084+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15c800de30 con 0x7f15d410a980 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.084+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41a6540 con 0x7f15d410a980 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.084+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8005240 con 0x7f15d410a980 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.084+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41ade60 con 0x7f15d410a980 2024-05-27T07:00:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.085+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d410c070 con 0x7f15d410a980 2024-05-27T07:00:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.088+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15c80049a0 con 0x7f15d410a980 2024-05-27T07:00:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.090+0000 7f15d2000700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 0x7f15bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.091+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f15c80e21b0 con 0x7f15d410a980 2024-05-27T07:00:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.091+0000 7f15d9200700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 0x7f15bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.092+0000 7f15d9200700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 0x7f15bc0b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f15c000b5c0 tx=0x7f15c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.094+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15c801f070 con 0x7f15d410a980 2024-05-27T07:00:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.385+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d402e5e0 con 0x7f15d410a980 2024-05-27T07:00:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.387+0000 7f15d2000700 1 -- 172.21.15.16:0/719133312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11926 (secure 0 0 0) 0x7f15c80a9ac0 con 0x7f15d410a980 2024-05-27T07:00:16.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:16.380 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T06:56:48.860392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34517,34478,34451,24481,34421]},"id":1}]} 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.390+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 msgr2=0x7f15bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 0x7f15bc0b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f15c000b5c0 tx=0x7f15c0005960 comp rx=0 tx=0).stop 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 msgr2=0x7f15d41a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f15c800d950 tx=0x7f15c800dd10 comp rx=0 tx=0).stop 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 shutdown_connections 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f15bc0b2880 0x7f15bc0b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d4110960 0x7f15d41a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d410a980 0x7f15d41a1380 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 --2- 172.21.15.16:0/719133312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d410b2d0 0x7f15d41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 >> 172.21.15.16:0/719133312 conn(0x7f15d4077c70 msgr2=0x7f15d407b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 shutdown_connections 2024-05-27T07:00:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:16.391+0000 7f15dc600700 1 -- 172.21.15.16:0/719133312 wait complete. 2024-05-27T07:00:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-05-27T07:00:17.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:17 smithi016 ceph-mon[31415]: pgmap v549: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:17.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:17 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/719133312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:17.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:00:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:17 smithi164 ceph-mon[35327]: pgmap v549: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:17.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:17 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/719133312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:17.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:17 smithi069 ceph-mon[35276]: pgmap v549: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:17.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:17 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/719133312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:00:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:19 smithi164 ceph-mon[35327]: pgmap v550: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:19 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:19 smithi016 ceph-mon[31415]: pgmap v550: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:19 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:19 smithi069 ceph-mon[35276]: pgmap v550: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:19 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 -- 172.21.15.16:0/805640810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106eb0 msgr2=0x7fc548113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 --2- 172.21.15.16:0/805640810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106eb0 0x7fc548113340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc54000b0a0 tx=0x7fc54000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 -- 172.21.15.16:0/805640810 shutdown_connections 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 --2- 172.21.15.16:0/805640810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106eb0 0x7fc548113340 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 --2- 172.21.15.16:0/805640810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54810c5a0 0x7fc54810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 --2- 172.21.15.16:0/805640810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc548106560 0x7fc548106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 -- 172.21.15.16:0/805640810 >> 172.21.15.16:0/805640810 conn(0x7fc548102040 msgr2=0x7fc548104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 -- 172.21.15.16:0/805640810 shutdown_connections 2024-05-27T07:00:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.325+0000 7fc54f400700 1 -- 172.21.15.16:0/805640810 wait complete. 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.326+0000 7fc54f400700 1 Processor -- start 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.326+0000 7fc54f400700 1 -- start start 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc548106eb0 0x7fc54819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 0x7fc5481a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481165d0 con 0x7fc548106560 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc548116450 con 0x7fc548106eb0 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54f400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc548116750 con 0x7fc54810c5a0 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48788/0 (socket says 172.21.15.16:48788) 2024-05-27T07:00:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.327+0000 7fc54ca00700 1 -- 172.21.15.16:0/552900429 learned_addr learned my addr 172.21.15.16:0/552900429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc547e00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc548106eb0 0x7fc54819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54d400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 0x7fc5481a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 -- 172.21.15.16:0/552900429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 msgr2=0x7fc5481a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 0x7fc5481a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 -- 172.21.15.16:0/552900429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc548106eb0 msgr2=0x7fc54819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc548106eb0 0x7fc54819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 -- 172.21.15.16:0/552900429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481a1dd0 con 0x7fc548106560 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54d400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 0x7fc5481a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.328+0000 7fc54ca00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e970 tx=0x7fc53c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.329+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00cbd0 con 0x7fc548106560 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.329+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc53c00cd70 con 0x7fc548106560 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.329+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a2060 con 0x7fc548106560 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.329+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005610 con 0x7fc548106560 2024-05-27T07:00:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.329+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a99e0 con 0x7fc548106560 2024-05-27T07:00:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.330+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5480512b0 con 0x7fc548106560 2024-05-27T07:00:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.333+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc53c004750 con 0x7fc548106560 2024-05-27T07:00:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.335+0000 7fc545600700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 0x7fc5300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.335+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc53c0e28d0 con 0x7fc548106560 2024-05-27T07:00:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.335+0000 7fc547e00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 0x7fc5300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.336+0000 7fc547e00700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 0x7fc5300b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc534005950 tx=0x7fc5340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.344+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc53c0aa1e0 con 0x7fc548106560 2024-05-27T07:00:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.585+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54819e260 con 0x7fc548106560 2024-05-27T07:00:20.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:00:20.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:20 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:00:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.589+0000 7fc545600700 1 -- 172.21.15.16:0/552900429 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24708 (secure 0 0 0) 0x7fc53c0a9ae0 con 0x7fc548106560 2024-05-27T07:00:20.591 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:00:20.591 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:20 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:00:20.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:20.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T06:56:40.328992+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:00:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 msgr2=0x7fc5300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 0x7fc5300b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc534005950 tx=0x7fc5340058e0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 msgr2=0x7fc54819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e970 tx=0x7fc53c00ed30 comp rx=0 tx=0).stop 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 shutdown_connections 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7fc5300b2900 0x7fc5300b4e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54810c5a0 0x7fc5481a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548106560 0x7fc54819cf00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 --2- 172.21.15.16:0/552900429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc548106eb0 0x7fc54819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.592+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 >> 172.21.15.16:0/552900429 conn(0x7fc548102040 msgr2=0x7fc548103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.593+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 shutdown_connections 2024-05-27T07:00:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:20.593+0000 7fc54f400700 1 -- 172.21.15.16:0/552900429 wait complete. 2024-05-27T07:00:20.625 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:00:20.625 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:20 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:00:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:21 smithi164 ceph-mon[35327]: pgmap v551: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:21 smithi164 ceph-mon[35327]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T07:00:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:21 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/552900429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:21.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:21 smithi016 ceph-mon[31415]: pgmap v551: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:21.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:21 smithi016 ceph-mon[31415]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T07:00:21.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:21 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/552900429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:21.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:21 smithi069 ceph-mon[35276]: pgmap v551: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:21 smithi069 ceph-mon[35276]: from='mgr.14152 172.21.15.16:0/2985527985' entity='mgr.x' 2024-05-27T07:00:21.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:21 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/552900429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:21.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T06:55:13.518131+0000', 'modified': '2024-05-27T06:56:48.860392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34478, 'mds_2': 34451, 'mds_3': 34421, 'mds_4': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/1261534836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 1261534836}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 1261534836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/1178863608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 1178863608}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 1178863608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/1294360695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 1294360695}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 1294360695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2731773245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2731773245}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2731773245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/162118671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 162118671}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 162118671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.164:6839/3910557712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 3910557712}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 3910557712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34421': {'gid': 34421, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 41, 'addr': '172.21.15.69:6833/123964468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6832', 'nonce': 123964468}, {'type': 'v1', 'addr': '172.21.15.69:6833', 'nonce': 123964468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.69:6835/4223174404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 4223174404}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 4223174404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34478': {'gid': 34478, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.69:6837/2624187710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2624187710}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2624187710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.69:6839/962503578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 962503578}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 962503578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34517, 34478, 34451, 24481, 34421]} 2024-05-27T07:00:21.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T07:00:21.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T07:00:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:23.563 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:23 smithi016 ceph-mon[31415]: pgmap v552: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:23.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:23 smithi069 ceph-mon[35276]: pgmap v552: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:23.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:23 smithi164 ceph-mon[35327]: pgmap v552: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:24.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 -- 172.21.15.16:0/939336337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69541054d0 msgr2=0x7f69541058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 --2- 172.21.15.16:0/939336337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69541054d0 0x7f69541058e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6948009990 tx=0x7f6948009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 -- 172.21.15.16:0/939336337 shutdown_connections 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 --2- 172.21.15.16:0/939336337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 0x7f6954110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 --2- 172.21.15.16:0/939336337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69541054d0 0x7f69541058e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 --2- 172.21.15.16:0/939336337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6954105e20 0x7f695410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 -- 172.21.15.16:0/939336337 >> 172.21.15.16:0/939336337 conn(0x7f69540fedb0 msgr2=0x7f69541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 -- 172.21.15.16:0/939336337 shutdown_connections 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.622+0000 7f695c200700 1 -- 172.21.15.16:0/939336337 wait complete. 2024-05-27T07:00:24.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.624+0000 7f695c200700 1 Processor -- start 2024-05-27T07:00:24.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.624+0000 7f695c200700 1 -- start start 2024-05-27T07:00:24.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69541054d0 0x7f695419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 0x7f695419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6954114320 con 0x7f6954105e20 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69541141a0 con 0x7f695410e900 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695c200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69541144a0 con 0x7f69541054d0 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f6958e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f6958e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57592/0 (socket says 172.21.15.16:57592) 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f6958e00700 1 -- 172.21.15.16:0/1745436039 learned_addr learned my addr 172.21.15.16:0/1745436039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f6959800700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69541054d0 0x7f695419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.625+0000 7f695a200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 0x7f695419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 -- 172.21.15.16:0/1745436039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69541054d0 msgr2=0x7f695419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69541054d0 0x7f695419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 -- 172.21.15.16:0/1745436039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 msgr2=0x7f695419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 0x7f695419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 -- 172.21.15.16:0/1745436039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695419fc20 con 0x7f6954105e20 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6958e00700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f694000d950 tx=0x7f694000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6952000700 1 -- 172.21.15.16:0/1745436039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940016070 con 0x7f6954105e20 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6952000700 1 -- 172.21.15.16:0/1745436039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6940010460 con 0x7f6954105e20 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f6952000700 1 -- 172.21.15.16:0/1745436039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694000f620 con 0x7f6954105e20 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.626+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695419ff10 con 0x7f6954105e20 2024-05-27T07:00:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.627+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a7830 con 0x7f6954105e20 2024-05-27T07:00:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.631+0000 7f6952000700 1 -- 172.21.15.16:0/1745436039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6940019070 con 0x7f6954105e20 2024-05-27T07:00:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 Processor -- start 2024-05-27T07:00:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 -- start start 2024-05-27T07:00:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea91170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3e9bedf0 con 0x560f3ea96ce0 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f3e9ae990 con 0x560f3ea90d60 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f696ad92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f3e9b29d0 con 0x560f3e911df0 2024-05-27T07:00:24.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.632+0000 7f6952000700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 0x7f693c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6952000700 1 -- 172.21.15.16:0/1745436039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f69400e21b0 con 0x7f6954105e20 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57604/0 (socket says 172.21.15.16:57604) 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 -- 172.21.15.16:0/940711382 learned_addr learned my addr 172.21.15.16:0/940711382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f695a200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 0x7f693c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6958e00700 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea91170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f695a200700 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 -- 172.21.15.16:0/940711382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 msgr2=0x560f3ea967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 -- 172.21.15.16:0/940711382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 msgr2=0x560f3ea91170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea91170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.633+0000 7f6959800700 1 -- 172.21.15.16:0/940711382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3e9773d0 con 0x560f3ea96ce0 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.634+0000 7f695a200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 0x7f693c0b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560f3ea91ae0 tx=0x7f694c009070 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.634+0000 7f6959800700 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f69480096a0 tx=0x7f69480096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e223c1b7aa7a9fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.634+0000 7f6946a00700 1 -- 172.21.15.16:0/940711382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948005070 con 0x560f3ea96ce0 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.634+0000 7f6946a00700 1 -- 172.21.15.16:0/940711382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69480056a0 con 0x560f3ea96ce0 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.634+0000 7f6946a00700 1 -- 172.21.15.16:0/940711382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694800ba30 con 0x560f3ea96ce0 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.635+0000 7f696ad92b80 1 -- 172.21.15.16:0/940711382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 msgr2=0x560f3ea90820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.635+0000 7f696ad92b80 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f69480096a0 tx=0x7f69480096d0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.635+0000 7f696ad92b80 1 -- 172.21.15.16:0/940711382 shutdown_connections 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea91170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 --2- 172.21.15.16:0/940711382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea90820 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 -- 172.21.15.16:0/940711382 >> 172.21.15.16:0/940711382 conn(0x560f3e971f70 msgr2=0x560f3ea8b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 -- 172.21.15.16:0/940711382 shutdown_connections 2024-05-27T07:00:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 -- 172.21.15.16:0/940711382 wait complete. 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.636+0000 7f696ad92b80 1 -- start start 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 Processor -- start 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea9aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3e8b4f50 con 0x560f3ea96ce0 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f3e94f710 con 0x560f3ea90d60 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f3ea9d750 con 0x560f3e911df0 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.637+0000 7f696ad92b80 10 client.0 _ll_register_callbacks cb 0x560f3e958c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57618/0 (socket says 172.21.15.16:57618) 2024-05-27T07:00:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 -- 172.21.15.16:0/2288736091 learned_addr learned my addr 172.21.15.16:0/2288736091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f6959800700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f6958e00700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea9aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 -- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 msgr2=0x560f3ea9a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea9a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 -- 172.21.15.16:0/2288736091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 msgr2=0x560f3ea9aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea9aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.638+0000 7f695a200700 1 -- 172.21.15.16:0/2288736091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3e8fe880 con 0x560f3ea96ce0 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f695a200700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f694c019a60 tx=0x7f694c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599495506d5cbd41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f693b400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c004bb0 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f694c012040 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69480056a0 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f694c02f050 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f696ad92b80 10 client.15201 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.639+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f3e97f010 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69480056a0 con 0x560f3ea96ce0 2024-05-27T07:00:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f694c024420 con 0x560f3ea96ce0 2024-05-27T07:00:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:00:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 20 client.15201 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eaaa7a0 0x560f3eaabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 4 client.15201 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:00:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.640+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3e9c0d40 con 0x560f3eaaa7a0 2024-05-27T07:00:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.641+0000 7f6959800700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eaaa7a0 0x560f3eaabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.642+0000 7f6959800700 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eaaa7a0 0x560f3eaabc10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf7486fcf8c54db5 server_cookie=f14cf06e1f499372 in_seq=0 out_seq=0 2024-05-27T07:00:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.642+0000 7f693b400700 10 client.15201 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:00:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f693b400700 1 -- 172.21.15.16:0/2288736091 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560f3e9c0d40 con 0x560f3eaaa7a0 2024-05-27T07:00:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f693b400700 10 client.15201 handle_command_reply: tid=0 2024-05-27T07:00:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 client.15201 shutdown 2024-05-27T07:00:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 msgr2=0x560f3ea9af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f694c019a60 tx=0x7f694c004b40 comp rx=0 tx=0).stop 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 shutdown_connections 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eaaa7a0 0x560f3eaabc10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9af90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea9a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 --2- 172.21.15.16:0/2288736091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea9aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f693b400700 0 client.15201 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:00:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 >> 172.21.15.16:0/2288736091 conn(0x560f3e971f70 msgr2=0x560f3ea8b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 shutdown_connections 2024-05-27T07:00:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.645+0000 7f696ad92b80 1 -- 172.21.15.16:0/2288736091 wait complete. 2024-05-27T07:00:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.646+0000 7f696ad92b80 20 client.15201 trim_cache size 0 max 16384 2024-05-27T07:00:24.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.669+0000 7f696ad92b80 1 Processor -- start 2024-05-27T07:00:24.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.669+0000 7f696ad92b80 1 -- start start 2024-05-27T07:00:24.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea95d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaa9f50 con 0x560f3ea96ce0 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaaa620 con 0x560f3ea90d60 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f696ad92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaaacf0 con 0x560f3e911df0 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f695a200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f695a200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57622/0 (socket says 172.21.15.16:57622) 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f695a200700 1 -- 172.21.15.16:0/1988191813 learned_addr learned my addr 172.21.15.16:0/1988191813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f6959800700 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.670+0000 7f6958e00700 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea95d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 -- 172.21.15.16:0/1988191813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 msgr2=0x560f3ea90820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea90820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 -- 172.21.15.16:0/1988191813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 msgr2=0x560f3ea95d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea95d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 -- 172.21.15.16:0/1988191813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3ea99830 con 0x560f3ea96ce0 2024-05-27T07:00:24.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f695a200700 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f694c0061e0 tx=0x7f694c024ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc9362d48e55e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f6946000700 1 -- 172.21.15.16:0/1988191813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c037020 con 0x560f3ea96ce0 2024-05-27T07:00:24.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f6946000700 1 -- 172.21.15.16:0/1988191813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c024420 con 0x560f3ea96ce0 2024-05-27T07:00:24.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.671+0000 7f6946000700 1 -- 172.21.15.16:0/1988191813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c014ad0 con 0x560f3ea96ce0 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.672+0000 7f696ad92b80 1 -- 172.21.15.16:0/1988191813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 msgr2=0x560f3ea9c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.672+0000 7f696ad92b80 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f694c0061e0 tx=0x7f694c024ce0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 -- 172.21.15.16:0/1988191813 shutdown_connections 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c980 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3e911df0 0x560f3ea90820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 --2- 172.21.15.16:0/1988191813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3ea90d60 0x560f3ea95d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 -- 172.21.15.16:0/1988191813 >> 172.21.15.16:0/1988191813 conn(0x560f3e971f70 msgr2=0x560f3ea95660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 -- 172.21.15.16:0/1988191813 shutdown_connections 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 -- 172.21.15.16:0/1988191813 wait complete. 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 -- start start 2024-05-27T07:00:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.673+0000 7f696ad92b80 1 Processor -- start 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3e911df0 0x560f3ea9bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaaba90 con 0x560f3ea90d60 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaab3c0 con 0x560f3e911df0 2024-05-27T07:00:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f3eaac160 con 0x560f3ea96ce0 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f696ad92b80 10 client.0 _ll_register_callbacks cb 0x560f3e958c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f6958e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f6958e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57630/0 (socket says 172.21.15.16:57630) 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.674+0000 7f6958e00700 1 -- 172.21.15.16:0/4061619640 learned_addr learned my addr 172.21.15.16:0/4061619640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6959800700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3e911df0 0x560f3ea9bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f695a200700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 -- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 msgr2=0x560f3ea9c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 -- 172.21.15.16:0/4061619640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3e911df0 msgr2=0x560f3ea9bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3e911df0 0x560f3ea9bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 -- 172.21.15.16:0/4061619640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3eab3400 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f695a200700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f6958e00700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f69400a9350 tx=0x7f694000dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6465386c51234271 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f693b400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.675+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400c3060 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694c024420 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f69400a9830 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6940020070 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 10 client.15207 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f3eb1e920 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c024420 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f69400b1d90 con 0x560f3ea90d60 2024-05-27T07:00:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 10 client.15207 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:00:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 20 client.15207 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.676+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eb20d10 0x560f3eb22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.677+0000 7f696ad92b80 4 client.15207 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:00:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.677+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "status"}) -- 0x560f3e9ba520 con 0x560f3eb20d10 2024-05-27T07:00:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.677+0000 7f6959800700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eb20d10 0x560f3eb22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.678+0000 7f6959800700 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eb20d10 0x560f3eb22180 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6dc3c36ae7ca7b9 server_cookie=e4a9e21087cd86f5 in_seq=0 out_seq=0 2024-05-27T07:00:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.678+0000 7f693b400700 10 client.15207 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:00:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f693b400700 1 -- 172.21.15.16:0/4061619640 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560f3e9ba520 con 0x560f3eb20d10 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f693b400700 10 client.15207 handle_command_reply: tid=0 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 client.15207 shutdown 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 msgr2=0x560f3ea9c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f69400a9350 tx=0x7f694000dfa0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 shutdown_connections 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x560f3eb20d10 0x560f3eb22180 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f3ea96ce0 0x560f3ea9c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f3e911df0 0x560f3ea9bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 --2- 172.21.15.16:0/4061619640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3ea90d60 0x560f3ea9c050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 >> 172.21.15.16:0/4061619640 conn(0x560f3e971f70 msgr2=0x560f3ea941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.679+0000 7f693b400700 0 client.15207 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.680+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 shutdown_connections 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.680+0000 7f696ad92b80 1 -- 172.21.15.16:0/4061619640 wait complete. 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.680+0000 7f696ad92b80 20 client.15207 trim_cache size 0 max 16384 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T07:00:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 222.268806345, 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 350.04256458999998 2024-05-27T07:00:24.675 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.680+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 msgr2=0x7f693c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.680+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 0x7f693c0b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560f3ea91ae0 tx=0x7f694c009070 comp rx=0 tx=0).stop 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 msgr2=0x7f695419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f694000d950 tx=0x7f694000dd10 comp rx=0 tx=0).stop 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 shutdown_connections 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f693c0b2900 0x7f693c0b4e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695410e900 0x7f695419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69541054d0 0x7f695419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 --2- 172.21.15.16:0/1745436039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6954105e20 0x7f695419b2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 >> 172.21.15.16:0/1745436039 conn(0x7f69540fedb0 msgr2=0x7f69540ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 shutdown_connections 2024-05-27T07:00:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:24.681+0000 7f695c200700 1 -- 172.21.15.16:0/1745436039 wait complete. 2024-05-27T07:00:25.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:25 smithi016 ceph-mon[31415]: pgmap v553: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:25.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:25 smithi069 ceph-mon[35276]: pgmap v553: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:25.634 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:25 smithi164 ceph-mon[35327]: pgmap v553: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:25.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-05-27T07:00:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:27.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:27 smithi016 ceph-mon[31415]: pgmap v554: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:27.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:27 smithi069 ceph-mon[35276]: pgmap v554: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:27 smithi164 ceph-mon[35327]: pgmap v554: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 -- 172.21.15.16:0/3260351562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 msgr2=0x7f5e00106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 --2- 172.21.15.16:0/3260351562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e00106a20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5df00099c0 tx=0x7f5df0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 -- 172.21.15.16:0/3260351562 shutdown_connections 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 --2- 172.21.15.16:0/3260351562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 0x7f5e001131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 --2- 172.21.15.16:0/3260351562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e00106a20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 --2- 172.21.15.16:0/3260351562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 0x7f5e0010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 -- 172.21.15.16:0/3260351562 >> 172.21.15.16:0/3260351562 conn(0x7f5e00102020 msgr2=0x7f5e00104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 -- 172.21.15.16:0/3260351562 shutdown_connections 2024-05-27T07:00:28.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.796+0000 7f5dffe00700 1 -- 172.21.15.16:0/3260351562 wait complete. 2024-05-27T07:00:28.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.797+0000 7f5dffe00700 1 Processor -- start 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 -- start start 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 0x7f5e0019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 0x7f5e001a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001163c0 con 0x7f5e00106610 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00116240 con 0x7f5e0010b890 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dffe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00116540 con 0x7f5e0010af40 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57652/0 (socket says 172.21.15.16:57652) 2024-05-27T07:00:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.798+0000 7f5dfea00700 1 -- 172.21.15.16:0/2480160996 learned_addr learned my addr 172.21.15.16:0/2480160996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:28.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dff400700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 0x7f5e001a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5df6000700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 0x7f5e0019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 -- 172.21.15.16:0/2480160996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 msgr2=0x7f5e0019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 0x7f5e0019d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 -- 172.21.15.16:0/2480160996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 msgr2=0x7f5e001a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 0x7f5e001a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 -- 172.21.15.16:0/2480160996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a1c00 con 0x7f5e00106610 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.799+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5df0000c00 tx=0x7f5df000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.800+0000 7f5df7e00700 1 -- 172.21.15.16:0/2480160996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0027070 con 0x7f5e00106610 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.800+0000 7f5df7e00700 1 -- 172.21.15.16:0/2480160996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df000fe50 con 0x7f5e00106610 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.800+0000 7f5df7e00700 1 -- 172.21.15.16:0/2480160996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00173f0 con 0x7f5e00106610 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.800+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e001a1e90 con 0x7f5e00106610 2024-05-27T07:00:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.800+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001a2320 con 0x7f5e00106610 2024-05-27T07:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.804+0000 7f5df7e00700 1 -- 172.21.15.16:0/2480160996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5df002a070 con 0x7f5e00106610 2024-05-27T07:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.805+0000 7f5e1478eb80 1 Processor -- start 2024-05-27T07:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.805+0000 7f5e1478eb80 1 -- start start 2024-05-27T07:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.805+0000 7f5df7e00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 0x7f5de40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.805+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.805+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 0x55ee9280bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92817690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee92738df0 con 0x55ee927779a0 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee92728990 con 0x55ee9268bdf0 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee9272c9d0 con 0x55ee92815270 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5df7e00700 1 -- 172.21.15.16:0/2480160996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5df00e34e0 con 0x7f5e00106610 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dff400700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 0x7f5de40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57656/0 (socket says 172.21.15.16:57656) 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 -- 172.21.15.16:0/1913947661 learned_addr learned my addr 172.21.15.16:0/1913947661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5df6000700 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 0x55ee9280bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 -- 172.21.15.16:0/1913947661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 msgr2=0x55ee92817690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92817690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 -- 172.21.15.16:0/1913947661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 msgr2=0x55ee9280bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.806+0000 7f5dfea00700 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 0x55ee9280bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5dfea00700 1 -- 172.21.15.16:0/1913947661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee926f13d0 con 0x55ee927779a0 2024-05-27T07:00:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5df6000700 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 0x55ee9280bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5dff400700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 0x7f5de40b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d50 tx=0x7f5df8009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5dfea00700 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5df00aa710 tx=0x7f5df00aa660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc20346adc5675f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5defe00700 1 -- 172.21.15.16:0/1913947661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00aa780 con 0x55ee927779a0 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5defe00700 1 -- 172.21.15.16:0/1913947661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df00aeb30 con 0x55ee927779a0 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.807+0000 7f5defe00700 1 -- 172.21.15.16:0/1913947661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df005c0e0 con 0x55ee927779a0 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 -- 172.21.15.16:0/1913947661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 msgr2=0x55ee9280b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5df00aa710 tx=0x7f5df00aa660 comp rx=0 tx=0).stop 2024-05-27T07:00:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 -- 172.21.15.16:0/1913947661 shutdown_connections 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92817690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee927779a0 0x55ee9280b700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/1913947661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee9268bdf0 0x55ee9280bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 -- 172.21.15.16:0/1913947661 >> 172.21.15.16:0/1913947661 conn(0x55ee92725340 msgr2=0x55ee928066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.808+0000 7f5e1478eb80 1 -- 172.21.15.16:0/1913947661 shutdown_connections 2024-05-27T07:00:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.809+0000 7f5e1478eb80 1 -- 172.21.15.16:0/1913947661 wait complete. 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.809+0000 7f5e1478eb80 1 -- start start 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 Processor -- start 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee92812710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92812c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee9262ef50 con 0x55ee9268bdf0 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee926c9710 con 0x55ee927779a0 2024-05-27T07:00:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee92738df0 con 0x55ee92815270 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5e1478eb80 10 client.0 _ll_register_callbacks cb 0x55ee926d2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.810+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57666/0 (socket says 172.21.15.16:57666) 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 -- 172.21.15.16:0/3838668392 learned_addr learned my addr 172.21.15.16:0/3838668392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5df6000700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee92812710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dff400700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92812c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 -- 172.21.15.16:0/3838668392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 msgr2=0x55ee92812c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92812c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 -- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 msgr2=0x55ee92812710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee92812710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 -- 172.21.15.16:0/3838668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee92678880 con 0x55ee9268bdf0 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5df6000700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee92812710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5dfea00700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5df00a67b0 tx=0x7f5df00b3550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c33810ae1f5361 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.811+0000 7f5decc00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00aabf0 con 0x55ee9268bdf0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5df00e3190 con 0x55ee9268bdf0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df00aeb30 con 0x55ee9268bdf0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5df0057be0 con 0x55ee9268bdf0 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5e1478eb80 10 client.15219 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee926f9010 con 0x55ee9268bdf0 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df00aeb30 con 0x55ee9268bdf0 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.812+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5df00920d0 con 0x55ee9268bdf0 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 10 client.15219 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 20 client.15219 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee928245d0 0x55ee92825a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 4 client.15219 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:00:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee9273ad40 con 0x55ee928245d0 2024-05-27T07:00:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.813+0000 7f5df6000700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee928245d0 0x55ee92825a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.814+0000 7f5df6000700 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee928245d0 0x55ee92825a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3a895a8d353d564 server_cookie=e9c1b63fb244eec4 in_seq=0 out_seq=0 2024-05-27T07:00:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.814+0000 7f5decc00700 10 client.15219 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:00:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5decc00700 1 -- 172.21.15.16:0/3838668392 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee9273ad40 con 0x55ee928245d0 2024-05-27T07:00:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5decc00700 10 client.15219 handle_command_reply: tid=0 2024-05-27T07:00:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 client.15219 shutdown 2024-05-27T07:00:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 msgr2=0x55ee928121d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5df00a67b0 tx=0x7f5df00b3550 comp rx=0 tx=0).stop 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 shutdown_connections 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92812c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928121d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee928245d0 0x55ee92825a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/3838668392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee92812710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 >> 172.21.15.16:0/3838668392 conn(0x55ee92725340 msgr2=0x55ee928062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5decc00700 0 client.15219 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.817+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 shutdown_connections 2024-05-27T07:00:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.818+0000 7f5e1478eb80 1 -- 172.21.15.16:0/3838668392 wait complete. 2024-05-27T07:00:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.818+0000 7f5e1478eb80 20 client.15219 trim_cache size 0 max 16384 2024-05-27T07:00:28.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.841+0000 7f5e1478eb80 1 Processor -- start 2024-05-27T07:00:28.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.841+0000 7f5e1478eb80 1 -- start start 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92823650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee92823d80 con 0x55ee9268bdf0 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee92824450 con 0x55ee927779a0 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee92824b20 con 0x55ee92815270 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5dfea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57676/0 (socket says 172.21.15.16:57676) 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5dfea00700 1 -- 172.21.15.16:0/2037859240 learned_addr learned my addr 172.21.15.16:0/2037859240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.842+0000 7f5df6000700 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dff400700 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92823650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 -- 172.21.15.16:0/2037859240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 msgr2=0x55ee92823650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92823650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 -- 172.21.15.16:0/2037859240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 msgr2=0x55ee928114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 -- 172.21.15.16:0/2037859240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee92807ba0 con 0x55ee9268bdf0 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5df00aa710 tx=0x7f5df00cd180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fdd8f2a36898fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.843+0000 7f5def400700 1 -- 172.21.15.16:0/2037859240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00aeb30 con 0x55ee9268bdf0 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.844+0000 7f5def400700 1 -- 172.21.15.16:0/2037859240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df00aabf0 con 0x55ee9268bdf0 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.844+0000 7f5def400700 1 -- 172.21.15.16:0/2037859240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00920d0 con 0x55ee9268bdf0 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.844+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2037859240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 msgr2=0x55ee928066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.844+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5df00aa710 tx=0x7f5df00cd180 comp rx=0 tx=0).stop 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2037859240 shutdown_connections 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee92815270 0x55ee92823650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee9268bdf0 0x55ee928066f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2037859240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2037859240 >> 172.21.15.16:0/2037859240 conn(0x55ee92725340 msgr2=0x55ee9280b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2037859240 shutdown_connections 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2037859240 wait complete. 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 -- start start 2024-05-27T07:00:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.845+0000 7f5e1478eb80 1 Processor -- start 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee9268bdf0 0x55ee9282f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee928258c0 con 0x55ee92815270 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee928251f0 con 0x55ee927779a0 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee92825f90 con 0x55ee9268bdf0 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5e1478eb80 10 client.0 _ll_register_callbacks cb 0x55ee926d2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5dff400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5dff400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57690/0 (socket says 172.21.15.16:57690) 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.846+0000 7f5dff400700 1 -- 172.21.15.16:0/2254537940 learned_addr learned my addr 172.21.15.16:0/2254537940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee9268bdf0 0x55ee9282f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5df6000700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 -- 172.21.15.16:0/2254537940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee9268bdf0 msgr2=0x55ee9282f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee9268bdf0 0x55ee9282f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 -- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 msgr2=0x55ee928287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 -- 172.21.15.16:0/2254537940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee928293b0 con 0x55ee92815270 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5dff400700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5df801ac10 tx=0x7f5df8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aa773c8c4abfbc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.847+0000 7f5decc00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8029060 con 0x55ee92815270 2024-05-27T07:00:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5df8016b10 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df00aabf0 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5df8030050 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5e1478eb80 10 client.15225 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.848+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee9282dd70 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df00aabf0 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5df8015070 con 0x55ee92815270 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 10 client.15225 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 20 client.15225 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee92892050 0x55ee928934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 4 client.15225 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.849+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "status"}) -- 0x55ee92734520 con 0x55ee92892050 2024-05-27T07:00:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.850+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee92892050 0x55ee928934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.850+0000 7f5dfea00700 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee92892050 0x55ee928934c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17d69bffb5dd721b server_cookie=a1689622ad11c03a in_seq=0 out_seq=0 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.850+0000 7f5decc00700 10 client.15225 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5decc00700 1 -- 172.21.15.16:0/2254537940 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ee92734520 con 0x55ee92892050 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5decc00700 10 client.15225 handle_command_reply: tid=0 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 client.15225 shutdown 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 msgr2=0x55ee92828d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5df801ac10 tx=0x7f5df8004b40 comp rx=0 tx=0).stop 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 shutdown_connections 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee92815270 0x55ee92828d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee9268bdf0 0x55ee9282f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55ee92892050 0x55ee928934c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 --2- 172.21.15.16:0/2254537940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee927779a0 0x55ee928287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.851+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 >> 172.21.15.16:0/2254537940 conn(0x55ee92725340 msgr2=0x55ee9280a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.852+0000 7f5decc00700 0 client.15225 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:00:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.852+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 shutdown_connections 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.852+0000 7f5e1478eb80 1 -- 172.21.15.16:0/2254537940 wait complete. 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.852+0000 7f5e1478eb80 20 client.15225 trim_cache size 0 max 16384 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34421, 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T07:00:28.845 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 227.459118233, 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 385.65485952500001 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.852+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 msgr2=0x7f5de40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 0x7f5de40b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d50 tx=0x7f5df8009380 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 msgr2=0x7f5e0019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5df0000c00 tx=0x7f5df000f510 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 shutdown_connections 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5de40b2900 0x7f5de40b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0010b890 0x7f5e001a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e00106610 0x7f5e0019cd30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 --2- 172.21.15.16:0/2480160996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0010af40 0x7f5e0019d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 >> 172.21.15.16:0/2480160996 conn(0x7f5e00102020 msgr2=0x7f5e0010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 shutdown_connections 2024-05-27T07:00:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:28.853+0000 7f5dffe00700 1 -- 172.21.15.16:0/2480160996 wait complete. 2024-05-27T07:00:29.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:29 smithi016 ceph-mon[31415]: pgmap v555: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:29.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:29 smithi069 ceph-mon[35276]: pgmap v555: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:29.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T07:00:29.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:29 smithi164 ceph-mon[35327]: pgmap v555: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:31.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:31 smithi069 ceph-mon[35276]: pgmap v556: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:31.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:31 smithi164 ceph-mon[35327]: pgmap v556: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:31 smithi016 ceph-mon[31415]: pgmap v556: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 -- 172.21.15.16:0/3017235803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 msgr2=0x7f96081058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 --2- 172.21.15.16:0/3017235803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f96081058f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009990 tx=0x7f95f8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 -- 172.21.15.16:0/3017235803 shutdown_connections 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 --2- 172.21.15.16:0/3017235803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 0x7f9608110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 --2- 172.21.15.16:0/3017235803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f96081058f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 --2- 172.21.15.16:0/3017235803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 0x7f960810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 -- 172.21.15.16:0/3017235803 >> 172.21.15.16:0/3017235803 conn(0x7f9608078e30 msgr2=0x7f9608079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 -- 172.21.15.16:0/3017235803 shutdown_connections 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.826+0000 7f960d200700 1 -- 172.21.15.16:0/3017235803 wait complete. 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.827+0000 7f960d200700 1 Processor -- start 2024-05-27T07:00:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.827+0000 7f960d200700 1 -- start start 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 0x7f960819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 0x7f96081a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9608114410 con 0x7f96081054e0 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9608114290 con 0x7f960810e910 2024-05-27T07:00:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f960d200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9608114590 con 0x7f9608105e30 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f9605e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f9605e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42458/0 (socket says 172.21.15.16:42458) 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.828+0000 7f9605e00700 1 -- 172.21.15.16:0/1664230147 learned_addr learned my addr 172.21.15.16:0/1664230147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605400700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 0x7f960819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9606800700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 0x7f96081a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 -- 172.21.15.16:0/1664230147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 msgr2=0x7f960819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 0x7f960819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 -- 172.21.15.16:0/1664230147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 msgr2=0x7f96081a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 0x7f96081a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 -- 172.21.15.16:0/1664230147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96081a1f10 con 0x7f96081054e0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.829+0000 7f9605e00700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f95f80057f0 tx=0x7f95f8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.830+0000 7f95fea00700 1 -- 172.21.15.16:0/1664230147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8027070 con 0x7f96081054e0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.830+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96081a21a0 con 0x7f96081054e0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.830+0000 7f95fea00700 1 -- 172.21.15.16:0/1664230147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95f800fd60 con 0x7f96081054e0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.830+0000 7f95fea00700 1 -- 172.21.15.16:0/1664230147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8022de0 con 0x7f96081054e0 2024-05-27T07:00:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.830+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a9b20 con 0x7f96081054e0 2024-05-27T07:00:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.836+0000 7f95fea00700 1 -- 172.21.15.16:0/1664230147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f95f802a070 con 0x7f96081054e0 2024-05-27T07:00:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.836+0000 7f961bd61b80 1 Processor -- start 2024-05-27T07:00:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 -- start start 2024-05-27T07:00:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 0x5650647482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x565064748820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x5650647550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565064676df0 con 0x565064748e20 2024-05-27T07:00:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565064666990 con 0x5650645c9df0 2024-05-27T07:00:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f961bd61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56506466a9d0 con 0x5650646b59a0 2024-05-27T07:00:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.837+0000 7f95fea00700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 0x7f95ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9606800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 0x5650647482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x565064748820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9606800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 0x5650647482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47230/0 (socket says 172.21.15.16:47230) 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9606800700 1 -- 172.21.15.16:0/4083469125 learned_addr learned my addr 172.21.15.16:0/4083469125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f95fea00700 1 -- 172.21.15.16:0/1664230147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f95f80e34e0 con 0x7f96081054e0 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x5650647550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9606800700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 0x7f95ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 -- 172.21.15.16:0/4083469125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 msgr2=0x5650647482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 0x5650647482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 -- 172.21.15.16:0/4083469125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 msgr2=0x565064748820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x565064748820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.838+0000 7f9605400700 1 -- 172.21.15.16:0/4083469125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506462f3d0 con 0x565064748e20 2024-05-27T07:00:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.839+0000 7f9605400700 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x5650647550c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007720 tx=0x7f95f4007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29615db349185c7c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.839+0000 7f95f3400700 1 -- 172.21.15.16:0/4083469125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4011590 con 0x565064748e20 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.839+0000 7f95f3400700 1 -- 172.21.15.16:0/4083469125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x565064748e20 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.839+0000 7f95f3400700 1 -- 172.21.15.16:0/4083469125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4012af0 con 0x565064748e20 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 -- 172.21.15.16:0/4083469125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 msgr2=0x5650647550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f9606800700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 0x7f95ec0b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96000070b0 tx=0x7f96000093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x5650647550c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007720 tx=0x7f95f4007a30 comp rx=0 tx=0).stop 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 -- 172.21.15.16:0/4083469125 shutdown_connections 2024-05-27T07:00:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x5650647550c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650646b59a0 0x5650647482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 --2- 172.21.15.16:0/4083469125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x565064748820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.840+0000 7f961bd61b80 1 -- 172.21.15.16:0/4083469125 >> 172.21.15.16:0/4083469125 conn(0x565064663340 msgr2=0x565064744470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.841+0000 7f961bd61b80 1 -- 172.21.15.16:0/4083469125 shutdown_connections 2024-05-27T07:00:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.841+0000 7f961bd61b80 1 -- 172.21.15.16:0/4083469125 wait complete. 2024-05-27T07:00:32.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.841+0000 7f961bd61b80 1 -- start start 2024-05-27T07:00:32.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.841+0000 7f961bd61b80 1 Processor -- start 2024-05-27T07:00:32.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650645c9df0 0x5650647526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 0x565064752c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56506456cf50 con 0x565064748e20 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565064607710 con 0x5650646b59a0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565064755b30 con 0x5650645c9df0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f961bd61b80 10 client.0 _ll_register_callbacks cb 0x565064610c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.842+0000 7f9606800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42472/0 (socket says 172.21.15.16:42472) 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 -- 172.21.15.16:0/1245235914 learned_addr learned my addr 172.21.15.16:0/1245235914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9605e00700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650645c9df0 0x5650647526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9605400700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 0x565064752c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 -- 172.21.15.16:0/1245235914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650645c9df0 msgr2=0x5650647526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650645c9df0 0x5650647526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 -- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 msgr2=0x565064752c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 0x565064752c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9606800700 1 -- 172.21.15.16:0/1245235914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650645b6880 con 0x565064748e20 2024-05-27T07:00:32.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.843+0000 7f9605400700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 0x565064752c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f9606800700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9600014ea0 tx=0x7f9600004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37e21ea8dd4264d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f95e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600028060 con 0x565064748e20 2024-05-27T07:00:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f960001a0d0 con 0x565064748e20 2024-05-27T07:00:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f4017070 con 0x565064748e20 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f960002f050 con 0x565064748e20 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f961bd61b80 10 client.15237 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.844+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565064637010 con 0x565064748e20 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x565064748e20 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f96000236b0 con 0x565064748e20 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 10 client.15237 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 20 client.15237 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x565064762bd0 0x565064764040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 4 client.15237 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.845+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565064678d40 con 0x565064762bd0 2024-05-27T07:00:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.846+0000 7f9605e00700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x565064762bd0 0x565064764040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.847+0000 7f9605e00700 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x565064762bd0 0x565064764040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31353961f5c0721a server_cookie=c0aede00ae29c3c7 in_seq=0 out_seq=0 2024-05-27T07:00:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.847+0000 7f95e7e00700 10 client.15237 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:00:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f95e7e00700 1 -- 172.21.15.16:0/1245235914 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565064678d40 con 0x565064762bd0 2024-05-27T07:00:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f95e7e00700 10 client.15237 handle_command_reply: tid=0 2024-05-27T07:00:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 client.15237 shutdown 2024-05-27T07:00:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 msgr2=0x565064753140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9600014ea0 tx=0x7f9600004be0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 shutdown_connections 2024-05-27T07:00:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x565064762bd0 0x565064764040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565064748e20 0x565064753140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650645c9df0 0x5650647526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 --2- 172.21.15.16:0/1245235914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650646b59a0 0x565064752c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 >> 172.21.15.16:0/1245235914 conn(0x565064663340 msgr2=0x5650647482c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.849+0000 7f95e7e00700 0 client.15237 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.850+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 shutdown_connections 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.850+0000 7f961bd61b80 1 -- 172.21.15.16:0/1245235914 wait complete. 2024-05-27T07:00:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.850+0000 7f961bd61b80 20 client.15237 trim_cache size 0 max 16384 2024-05-27T07:00:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 Processor -- start 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 -- start start 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x565064748770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565064762380 con 0x5650646b59a0 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565064762a50 con 0x5650645c9df0 2024-05-27T07:00:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.874+0000 7f961bd61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565064763120 con 0x565064748e20 2024-05-27T07:00:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.875+0000 7f9605400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.875+0000 7f9605400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42488/0 (socket says 172.21.15.16:42488) 2024-05-27T07:00:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.875+0000 7f9605400700 1 -- 172.21.15.16:0/3163597228 learned_addr learned my addr 172.21.15.16:0/3163597228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.875+0000 7f9606800700 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x565064748770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.875+0000 7f9605e00700 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 -- 172.21.15.16:0/3163597228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 msgr2=0x565064748770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x565064748770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 -- 172.21.15.16:0/3163597228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 msgr2=0x56506474dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 -- 172.21.15.16:0/3163597228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565064745970 con 0x5650646b59a0 2024-05-27T07:00:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f9605400700 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c00 tx=0x7f95f4012420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9240063509e86e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f95f2a00700 1 -- 172.21.15.16:0/3163597228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4012740 con 0x5650646b59a0 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f95f2a00700 1 -- 172.21.15.16:0/3163597228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95f4017040 con 0x5650646b59a0 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.876+0000 7f95f2a00700 1 -- 172.21.15.16:0/3163597228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4011d20 con 0x5650646b59a0 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.877+0000 7f961bd61b80 1 -- 172.21.15.16:0/3163597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 msgr2=0x565064751620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.877+0000 7f961bd61b80 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c00 tx=0x7f95f4012420 comp rx=0 tx=0).stop 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 -- 172.21.15.16:0/3163597228 shutdown_connections 2024-05-27T07:00:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x565064748770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 --2- 172.21.15.16:0/3163597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x565064751620 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 -- 172.21.15.16:0/3163597228 >> 172.21.15.16:0/3163597228 conn(0x565064663340 msgr2=0x565064748060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 -- 172.21.15.16:0/3163597228 shutdown_connections 2024-05-27T07:00:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 -- 172.21.15.16:0/3163597228 wait complete. 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.878+0000 7f961bd61b80 1 -- start start 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 Processor -- start 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x56506474dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565064763ec0 con 0x5650646b59a0 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5650647637f0 con 0x5650645c9df0 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565064764590 con 0x565064748e20 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.879+0000 7f961bd61b80 10 client.0 _ll_register_callbacks cb 0x565064610c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42498/0 (socket says 172.21.15.16:42498) 2024-05-27T07:00:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 -- 172.21.15.16:0/2581706507 learned_addr learned my addr 172.21.15.16:0/2581706507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605e00700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9606800700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x56506474dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 -- 172.21.15.16:0/2581706507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 msgr2=0x56506474dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x56506474dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 -- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 msgr2=0x56506474d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.880+0000 7f9605400700 1 -- 172.21.15.16:0/2581706507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650647534b0 con 0x5650646b59a0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f9605400700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95f4006d60 tx=0x7f95f40116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff8aae19d5691f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f95e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4011c70 con 0x5650646b59a0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f4017040 con 0x5650646b59a0 2024-05-27T07:00:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f95f400f040 con 0x5650646b59a0 2024-05-27T07:00:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95f403b050 con 0x5650646b59a0 2024-05-27T07:00:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f961bd61b80 10 client.15243 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.881+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650647ce4b0 con 0x5650646b59a0 2024-05-27T07:00:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95f4017040 con 0x5650646b59a0 2024-05-27T07:00:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f95f4036030 con 0x5650646b59a0 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 10 client.15243 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 20 client.15243 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5650647d0260 0x5650647d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 4 client.15243 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.882+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "status"}) -- 0x565064672520 con 0x5650647d0260 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.883+0000 7f9605e00700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5650647d0260 0x5650647d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f95e7e00700 10 client.15243 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:00:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f9605e00700 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5650647d0260 0x5650647d16d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb8b6a788dcc447d server_cookie=68ec095e9d07c4af in_seq=0 out_seq=0 2024-05-27T07:00:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f95e7e00700 1 -- 172.21.15.16:0/2581706507 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565064672520 con 0x5650647d0260 2024-05-27T07:00:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f95e7e00700 10 client.15243 handle_command_reply: tid=0 2024-05-27T07:00:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f961bd61b80 1 client.15243 shutdown 2024-05-27T07:00:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 msgr2=0x56506474d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.884+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95f4006d60 tx=0x7f95f40116f0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 shutdown_connections 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5650647d0260 0x5650647d16d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565064748e20 0x56506474dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650645c9df0 0x56506474d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 --2- 172.21.15.16:0/2581706507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650646b59a0 0x56506474d830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 >> 172.21.15.16:0/2581706507 conn(0x565064663340 msgr2=0x565064749e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f95e7e00700 0 client.15243 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 shutdown_connections 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 1 -- 172.21.15.16:0/2581706507 wait complete. 2024-05-27T07:00:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.885+0000 7f961bd61b80 20 client.15243 trim_cache size 0 max 16384 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34451, 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 232.50912496199999, 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T07:00:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 375.15870559899997 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 msgr2=0x7f95ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 0x7f95ec0b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96000070b0 tx=0x7f96000093d0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 msgr2=0x7f960819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f95f80057f0 tx=0x7f95f8009f90 comp rx=0 tx=0).stop 2024-05-27T07:00:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 shutdown_connections 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f95ec0b2900 0x7f95ec0b4e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f960810e910 0x7f96081a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96081054e0 0x7f960819d040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 --2- 172.21.15.16:0/1664230147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9608105e30 0x7f960819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 >> 172.21.15.16:0/1664230147 conn(0x7f9608078e30 msgr2=0x7f9608103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 shutdown_connections 2024-05-27T07:00:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:32.886+0000 7f960d200700 1 -- 172.21.15.16:0/1664230147 wait complete. 2024-05-27T07:00:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:33 smithi016 ceph-mon[31415]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:33.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:33 smithi164 ceph-mon[35327]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:33.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:33 smithi069 ceph-mon[35276]: pgmap v557: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:33.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T07:00:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:35.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:35 smithi016 ceph-mon[31415]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:35.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:35 smithi164 ceph-mon[35327]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:35.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:35 smithi069 ceph-mon[35276]: pgmap v558: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-27T07:00:37.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 -- 172.21.15.16:0/3665010199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 msgr2=0x7f2508077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 --2- 172.21.15.16:0/3665010199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f2508077010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b0a0 tx=0x7f24f800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 -- 172.21.15.16:0/3665010199 shutdown_connections 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 --2- 172.21.15.16:0/3665010199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508110bd0 0x7f2508112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 --2- 172.21.15.16:0/3665010199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f2508077010 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 --2- 172.21.15.16:0/3665010199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508077550 0x7f2508110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.010+0000 7f250e800700 1 -- 172.21.15.16:0/3665010199 >> 172.21.15.16:0/3665010199 conn(0x7f25080ffe90 msgr2=0x7f25081022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.011+0000 7f250e800700 1 -- 172.21.15.16:0/3665010199 shutdown_connections 2024-05-27T07:00:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.011+0000 7f250e800700 1 -- 172.21.15.16:0/3665010199 wait complete. 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.012+0000 7f250e800700 1 Processor -- start 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.012+0000 7f250e800700 1 -- start start 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 0x7f250819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508110bd0 0x7f25081a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2508116560 con 0x7f2508076c00 2024-05-27T07:00:37.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25081163e0 con 0x7f2508110bd0 2024-05-27T07:00:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f250e800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f25081166e0 con 0x7f2508077550 2024-05-27T07:00:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42520/0 (socket says 172.21.15.16:42520) 2024-05-27T07:00:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 -- 172.21.15.16:0/1455104028 learned_addr learned my addr 172.21.15.16:0/1455104028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507e00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508110bd0 0x7f25081a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f24fea00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 0x7f250819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 -- 172.21.15.16:0/1455104028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 msgr2=0x7f250819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 0x7f250819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 -- 172.21.15.16:0/1455104028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508110bd0 msgr2=0x7f25081a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508110bd0 0x7f25081a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f2507400700 1 -- 172.21.15.16:0/1455104028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25081a1dc0 con 0x7f2508076c00 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.013+0000 7f24fea00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 0x7f250819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f2507400700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24f80062a0 tx=0x7f24f8004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f2504c00700 1 -- 172.21.15.16:0/1455104028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8021070 con 0x7f2508076c00 2024-05-27T07:00:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f2504c00700 1 -- 172.21.15.16:0/1455104028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f8007d40 con 0x7f2508076c00 2024-05-27T07:00:37.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25081a2050 con 0x7f2508076c00 2024-05-27T07:00:37.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f2504c00700 1 -- 172.21.15.16:0/1455104028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80052a0 con 0x7f2508076c00 2024-05-27T07:00:37.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.014+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081a2530 con 0x7f2508076c00 2024-05-27T07:00:37.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.018+0000 7f2504c00700 1 -- 172.21.15.16:0/1455104028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f24f8024070 con 0x7f2508076c00 2024-05-27T07:00:37.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.021+0000 7f251d348b80 1 Processor -- start 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.021+0000 7f251d348b80 1 -- start start 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.021+0000 7f2504c00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 0x7f24ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.021+0000 7f2504c00700 1 -- 172.21.15.16:0/1455104028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f24f800f020 con 0x7f2508076c00 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.021+0000 7f2507e00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 0x7f24ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067a9f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067aa0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0679cddf0 con 0x55a067920df0 2024-05-27T07:00:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a0679bd990 con 0x55a067aa5ce0 2024-05-27T07:00:37.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f251d348b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a0679c19d0 con 0x55a067a9fd60 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42522/0 (socket says 172.21.15.16:42522) 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f2507400700 1 -- 172.21.15.16:0/3619839245 learned_addr learned my addr 172.21.15.16:0/3619839245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f2507e00700 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067aa0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.022+0000 7f24fea00700 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067a9f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 -- 172.21.15.16:0/3619839245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 msgr2=0x55a067aa0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067aa0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507e00700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 0x7f24ec0b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2508077be0 tx=0x7f250000bed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 -- 172.21.15.16:0/3619839245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 msgr2=0x55a067a9f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067a9f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 -- 172.21.15.16:0/3619839245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0679863d0 con 0x55a067920df0 2024-05-27T07:00:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f2507400700 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a6f70 tx=0x7f24f8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed465d119800456 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f24fcc00700 1 -- 172.21.15.16:0/3619839245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80af2a0 con 0x55a067920df0 2024-05-27T07:00:37.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f24fcc00700 1 -- 172.21.15.16:0/3619839245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f80b3cc0 con 0x55a067920df0 2024-05-27T07:00:37.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.023+0000 7f24fcc00700 1 -- 172.21.15.16:0/3619839245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80ab3b0 con 0x55a067920df0 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 -- 172.21.15.16:0/3619839245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 msgr2=0x55a067aa57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a6f70 tx=0x7f24f8009b80 comp rx=0 tx=0).stop 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 -- 172.21.15.16:0/3619839245 shutdown_connections 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067aa0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067aa57a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 --2- 172.21.15.16:0/3619839245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067a9f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.024+0000 7f251d348b80 1 -- 172.21.15.16:0/3619839245 >> 172.21.15.16:0/3619839245 conn(0x55a067980f70 msgr2=0x55a067a9a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.025+0000 7f251d348b80 1 -- 172.21.15.16:0/3619839245 shutdown_connections 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.025+0000 7f251d348b80 1 -- 172.21.15.16:0/3619839245 wait complete. 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.025+0000 7f251d348b80 1 -- start start 2024-05-27T07:00:37.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.025+0000 7f251d348b80 1 Processor -- start 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067920df0 0x55a067aa94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 0x55a067aa9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0678c3f50 con 0x55a067a9fd60 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a06795e710 con 0x55a067920df0 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a067aac570 con 0x55a067aa5ce0 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f251d348b80 10 client.0 _ll_register_callbacks cb 0x55a067967c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f24fea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.026+0000 7f24fea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42526/0 (socket says 172.21.15.16:42526) 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 -- 172.21.15.16:0/2096075182 learned_addr learned my addr 172.21.15.16:0/2096075182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f2507400700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067920df0 0x55a067aa94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f2507e00700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 0x55a067aa9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 -- 172.21.15.16:0/2096075182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 msgr2=0x55a067aa9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 0x55a067aa9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 -- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067920df0 msgr2=0x55a067aa94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067920df0 0x55a067aa94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 -- 172.21.15.16:0/2096075182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06790d880 con 0x55a067a9fd60 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f2507e00700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 0x55a067aa9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24fea00700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f24f00106a0 tx=0x7f24f0010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6aebfeedc2e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.027+0000 7f24f5600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0016070 con 0x55a067a9fd60 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f24f0004220 con 0x55a067a9fd60 2024-05-27T07:00:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f80b3cc0 con 0x55a067a9fd60 2024-05-27T07:00:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24f0022050 con 0x55a067a9fd60 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f251d348b80 10 client.15255 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a06798e010 con 0x55a067a9fd60 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.028+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f80b3cc0 con 0x55a067a9fd60 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24f000f040 con 0x55a067a9fd60 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 10 client.15255 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 10 client.15255 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 20 client.15255 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067ab94a0 0x55a067aba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 4 client.15255 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.029+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0679cfd40 con 0x55a067ab94a0 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.030+0000 7f2507400700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067ab94a0 0x55a067aba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.031+0000 7f2507400700 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067ab94a0 0x55a067aba910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6dcf0a36b05091e server_cookie=12e2647f4edea07 in_seq=0 out_seq=0 2024-05-27T07:00:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.031+0000 7f24f5600700 10 client.15255 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:00:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f24f5600700 1 -- 172.21.15.16:0/2096075182 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a0679cfd40 con 0x55a067ab94a0 2024-05-27T07:00:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f24f5600700 10 client.15255 handle_command_reply: tid=0 2024-05-27T07:00:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 client.15255 shutdown 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 msgr2=0x55a067aa9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f24f00106a0 tx=0x7f24f0010a60 comp rx=0 tx=0).stop 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 shutdown_connections 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067ab94a0 0x55a067aba910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067aa5ce0 0x55a067aa9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067920df0 0x55a067aa94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 --2- 172.21.15.16:0/2096075182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067a9fd60 0x55a067aa9a20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 >> 172.21.15.16:0/2096075182 conn(0x55a067980f70 msgr2=0x55a067a9a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.033+0000 7f24f5600700 0 client.15255 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.034+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 shutdown_connections 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.034+0000 7f251d348b80 1 -- 172.21.15.16:0/2096075182 wait complete. 2024-05-27T07:00:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.034+0000 7f251d348b80 20 client.15255 trim_cache size 0 max 16384 2024-05-27T07:00:37.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.057+0000 7f251d348b80 1 Processor -- start 2024-05-27T07:00:37.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 -- start start 2024-05-27T07:00:37.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067920df0 0x55a067a9f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 0x55a067aa3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a067ab8c50 con 0x55a067aa5ce0 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a067ab9320 con 0x55a067a9fd60 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f251d348b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a067ab99f0 con 0x55a067920df0 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f2507e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f2507e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42532/0 (socket says 172.21.15.16:42532) 2024-05-27T07:00:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.058+0000 7f2507e00700 1 -- 172.21.15.16:0/3572572030 learned_addr learned my addr 172.21.15.16:0/3572572030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507400700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067920df0 0x55a067a9f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f24fea00700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 0x55a067aa3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 -- 172.21.15.16:0/3572572030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067920df0 msgr2=0x55a067a9f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067920df0 0x55a067a9f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 -- 172.21.15.16:0/3572572030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 msgr2=0x55a067aa3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 0x55a067aa3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 -- 172.21.15.16:0/3572572030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a067ab6ea0 con 0x55a067aa5ce0 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f24fea00700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 0x55a067aa3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.059+0000 7f2507e00700 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f250000b920 tx=0x7f2500014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f744752b834a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.060+0000 7f24f7e00700 1 -- 172.21.15.16:0/3572572030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500011e60 con 0x55a067aa5ce0 2024-05-27T07:00:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.060+0000 7f24f7e00700 1 -- 172.21.15.16:0/3572572030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2500023070 con 0x55a067aa5ce0 2024-05-27T07:00:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.060+0000 7f24f7e00700 1 -- 172.21.15.16:0/3572572030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250001f740 con 0x55a067aa5ce0 2024-05-27T07:00:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- 172.21.15.16:0/3572572030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 msgr2=0x55a067ab57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f250000b920 tx=0x7f2500014000 comp rx=0 tx=0).stop 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- 172.21.15.16:0/3572572030 shutdown_connections 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067aa5ce0 0x55a067ab57d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067920df0 0x55a067a9f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 --2- 172.21.15.16:0/3572572030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067a9fd60 0x55a067aa3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- 172.21.15.16:0/3572572030 >> 172.21.15.16:0/3572572030 conn(0x55a067980f70 msgr2=0x55a067aa92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- 172.21.15.16:0/3572572030 shutdown_connections 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- 172.21.15.16:0/3572572030 wait complete. 2024-05-27T07:00:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.061+0000 7f251d348b80 1 -- start start 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 Processor -- start 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067ac42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067abed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a067aba790 con 0x55a067920df0 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a067aba0c0 con 0x55a067aa5ce0 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a067abae60 con 0x55a067a9fd60 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.062+0000 7f251d348b80 10 client.0 _ll_register_callbacks cb 0x55a067967c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42538/0 (socket says 172.21.15.16:42538) 2024-05-27T07:00:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 -- 172.21.15.16:0/4053540278 learned_addr learned my addr 172.21.15.16:0/4053540278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507e00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067abed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f24fea00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067ac42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 -- 172.21.15.16:0/4053540278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 msgr2=0x55a067ac42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067ac42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 -- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 msgr2=0x55a067abed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067abed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 -- 172.21.15.16:0/4053540278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a067ab5910 con 0x55a067920df0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f24fea00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067ac42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507e00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067abed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f2507400700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b830 tx=0x7f24f80ab670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f7af9bb67d416 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.063+0000 7f24f5600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f806e070 con 0x55a067920df0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f24f80c5420 con 0x55a067920df0 2024-05-27T07:00:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2500023070 con 0x55a067920df0 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24f80b3cc0 con 0x55a067920df0 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f251d348b80 10 client.15261 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a067b24f70 con 0x55a067920df0 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.064+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2500023070 con 0x55a067920df0 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24f8011070 con 0x55a067920df0 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 10 client.15261 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 20 client.15261 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067b272d0 0x55a067b28740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 4 client.15261 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:00:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.065+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "status"}) -- 0x55a0679c9520 con 0x55a067b272d0 2024-05-27T07:00:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.066+0000 7f24fea00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067b272d0 0x55a067b28740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.067+0000 7f24fea00700 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067b272d0 0x55a067b28740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5563cb97c0090020 server_cookie=6033214df8d3f3d2 in_seq=0 out_seq=0 2024-05-27T07:00:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.067+0000 7f24f5600700 10 client.15261 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.067+0000 7f24f5600700 1 -- 172.21.15.16:0/4053540278 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a0679c9520 con 0x55a067b272d0 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.067+0000 7f24f5600700 10 client.15261 handle_command_reply: tid=0 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.067+0000 7f251d348b80 1 client.15261 shutdown 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 msgr2=0x55a067ac3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b830 tx=0x7f24f80ab670 comp rx=0 tx=0).stop 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 shutdown_connections 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55a067b272d0 0x55a067b28740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a067aa5ce0 0x55a067abed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a067920df0 0x55a067ac3d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 --2- 172.21.15.16:0/4053540278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a067a9fd60 0x55a067ac42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f24f5600700 0 client.15261 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:00:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 >> 172.21.15.16:0/4053540278 conn(0x55a067980f70 msgr2=0x55a067aa92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 shutdown_connections 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 1 -- 172.21.15.16:0/4053540278 wait complete. 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.068+0000 7f251d348b80 20 client.15261 trim_cache size 0 max 16384 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34478, 2024-05-27T07:00:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 237.717719061, 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 366.64731953400002 2024-05-27T07:00:37.063 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 msgr2=0x7f24ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 0x7f24ec0b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2508077be0 tx=0x7f250000bed0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 msgr2=0x7f250819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24f80062a0 tx=0x7f24f8004e80 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 shutdown_connections 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f24ec0b2880 0x7f24ec0b4d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2508110bd0 0x7f25081a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2508076c00 0x7f250819cef0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 --2- 172.21.15.16:0/1455104028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2508077550 0x7f250819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.069+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 >> 172.21.15.16:0/1455104028 conn(0x7f25080ffe90 msgr2=0x7f2508101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.070+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 shutdown_connections 2024-05-27T07:00:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:37.070+0000 7f250e800700 1 -- 172.21.15.16:0/1455104028 wait complete. 2024-05-27T07:00:37.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:37 smithi164 ceph-mon[35327]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:37.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:37 smithi016 ceph-mon[31415]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:37.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:37 smithi069 ceph-mon[35276]: pgmap v559: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:00:38.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:00:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:39.700 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:39 smithi016 ceph-mon[31415]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:39.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:39 smithi164 ceph-mon[35327]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:39.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:39 smithi069 ceph-mon[35276]: pgmap v560: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:00:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 -- 172.21.15.16:0/807627150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890110cc0 msgr2=0x7f38901130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 --2- 172.21.15.16:0/807627150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890110cc0 0x7f38901130e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f388800b0a0 tx=0x7f388800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 -- 172.21.15.16:0/807627150 shutdown_connections 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 --2- 172.21.15.16:0/807627150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890110cc0 0x7f38901130e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 --2- 172.21.15.16:0/807627150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 0x7f3890103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 --2- 172.21.15.16:0/807627150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890103a60 0x7f3890110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 -- 172.21.15.16:0/807627150 >> 172.21.15.16:0/807627150 conn(0x7f38900fee40 msgr2=0x7f3890101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.109+0000 7f3897e00700 1 -- 172.21.15.16:0/807627150 shutdown_connections 2024-05-27T07:00:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.110+0000 7f3897e00700 1 -- 172.21.15.16:0/807627150 wait complete. 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.111+0000 7f3897e00700 1 Processor -- start 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.111+0000 7f3897e00700 1 -- start start 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.111+0000 7f3897e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 0x7f389019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.111+0000 7f3897e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.111+0000 7f3897e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890110cc0 0x7f38901a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3897e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3890116590 con 0x7f3890103a60 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3897e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3890116410 con 0x7f3890110cc0 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3897e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3890116710 con 0x7f3890103110 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42568/0 (socket says 172.21.15.16:42568) 2024-05-27T07:00:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 -- 172.21.15.16:0/286906206 learned_addr learned my addr 172.21.15.16:0/286906206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3895e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890110cc0 0x7f38901a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3895400700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 0x7f389019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 -- 172.21.15.16:0/286906206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 msgr2=0x7f389019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 0x7f389019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 -- 172.21.15.16:0/286906206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890110cc0 msgr2=0x7f38901a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890110cc0 0x7f38901a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.112+0000 7f3894a00700 1 -- 172.21.15.16:0/286906206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901a1d50 con 0x7f3890103a60 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f3894a00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d980 tx=0x7f387c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f388e000700 1 -- 172.21.15.16:0/286906206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c017070 con 0x7f3890103a60 2024-05-27T07:00:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f388e000700 1 -- 172.21.15.16:0/286906206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f387c010460 con 0x7f3890103a60 2024-05-27T07:00:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f388e000700 1 -- 172.21.15.16:0/286906206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c00de60 con 0x7f3890103a60 2024-05-27T07:00:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901a2040 con 0x7f3890103a60 2024-05-27T07:00:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.113+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a9960 con 0x7f3890103a60 2024-05-27T07:00:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.117+0000 7f388e000700 1 -- 172.21.15.16:0/286906206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f387c010610 con 0x7f3890103a60 2024-05-27T07:00:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.118+0000 7f38a6a07b80 1 Processor -- start 2024-05-27T07:00:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.118+0000 7f38a6a07b80 1 -- start start 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 0x5636f6e330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636f6d54df0 con 0x5636f6ca7df0 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636f6d44990 con 0x5636f6e26e20 2024-05-27T07:00:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636f6d489d0 con 0x5636f6d939a0 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f388e000700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 0x7f38780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f3895400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f3895400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42584/0 (socket says 172.21.15.16:42584) 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f3895400700 1 -- 172.21.15.16:0/3683926027 learned_addr learned my addr 172.21.15.16:0/3683926027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f3894a00700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 0x5636f6e330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.119+0000 7f3895e00700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f388e000700 1 -- 172.21.15.16:0/286906206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f387c0e2070 con 0x7f3890103a60 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 0x7f38780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895400700 1 -- 172.21.15.16:0/3683926027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 msgr2=0x5636f6e262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895400700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895400700 1 -- 172.21.15.16:0/3683926027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 msgr2=0x5636f6e330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895400700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 0x5636f6e330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3895400700 1 -- 172.21.15.16:0/3683926027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f6d0d3d0 con 0x5636f6ca7df0 2024-05-27T07:00:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.120+0000 7f3894a00700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 0x5636f6e330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:00:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.121+0000 7f3895400700 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f388400bd70 tx=0x7f388400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d50b319250e21d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.121+0000 7f3895e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 0x7f38780b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5636f6e28fa0 tx=0x7f38880099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.121+0000 7f3882a00700 1 -- 172.21.15.16:0/3683926027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884015070 con 0x5636f6ca7df0 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.121+0000 7f3882a00700 1 -- 172.21.15.16:0/3683926027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f388400fda0 con 0x5636f6ca7df0 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.121+0000 7f3882a00700 1 -- 172.21.15.16:0/3683926027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840116a0 con 0x5636f6ca7df0 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 -- 172.21.15.16:0/3683926027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 msgr2=0x5636f6e26820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f388400bd70 tx=0x7f388400f640 comp rx=0 tx=0).stop 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 -- 172.21.15.16:0/3683926027 shutdown_connections 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6e26e20 0x5636f6e330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/3683926027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6ca7df0 0x5636f6e26820 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 -- 172.21.15.16:0/3683926027 >> 172.21.15.16:0/3683926027 conn(0x5636f6d41340 msgr2=0x5636f6e22470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 -- 172.21.15.16:0/3683926027 shutdown_connections 2024-05-27T07:00:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.122+0000 7f38a6a07b80 1 -- 172.21.15.16:0/3683926027 wait complete. 2024-05-27T07:00:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.123+0000 7f38a6a07b80 1 -- start start 2024-05-27T07:00:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.123+0000 7f38a6a07b80 1 Processor -- start 2024-05-27T07:00:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6ca7df0 0x5636f6e305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6d939a0 0x5636f6e30b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636f6c4af50 con 0x5636f6e26e20 2024-05-27T07:00:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636f6ce5710 con 0x5636f6d939a0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e33b30 con 0x5636f6ca7df0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f38a6a07b80 10 client.0 _ll_register_callbacks cb 0x5636f6ceec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f3895e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f3895e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42594/0 (socket says 172.21.15.16:42594) 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f3895e00700 1 -- 172.21.15.16:0/2710865533 learned_addr learned my addr 172.21.15.16:0/2710865533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f3895400700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6ca7df0 0x5636f6e305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.124+0000 7f3894a00700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6d939a0 0x5636f6e30b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 -- 172.21.15.16:0/2710865533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6ca7df0 msgr2=0x5636f6e305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6ca7df0 0x5636f6e305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 -- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6d939a0 msgr2=0x5636f6e30b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6d939a0 0x5636f6e30b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 -- 172.21.15.16:0/2710865533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f6c94880 con 0x5636f6e26e20 2024-05-27T07:00:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3895e00700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3888007870 tx=0x7f388800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2603cf4ef06140b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3877400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888032060 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f388801f440 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.125+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388400fda0 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.126+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3888039050 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.126+0000 7f38a6a07b80 10 client.15273 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.126+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636f6d15010 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.126+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f388400fda0 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.126+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3888011070 con 0x5636f6e26e20 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 10 client.15273 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:00:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 20 client.15273 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6e40aa0 0x5636f6e41f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 4 client.15273 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f6d56d40 con 0x5636f6e40aa0 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.127+0000 7f3895400700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6e40aa0 0x5636f6e41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.128+0000 7f3895400700 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6e40aa0 0x5636f6e41f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4448e2f309353fe2 server_cookie=3efaaae7f9eae2f6 in_seq=0 out_seq=0 2024-05-27T07:00:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.128+0000 7f3877400700 10 client.15273 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:00:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.130+0000 7f3877400700 1 -- 172.21.15.16:0/2710865533 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636f6d56d40 con 0x5636f6e40aa0 2024-05-27T07:00:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.130+0000 7f3877400700 10 client.15273 handle_command_reply: tid=0 2024-05-27T07:00:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 client.15273 shutdown 2024-05-27T07:00:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 msgr2=0x5636f6e31060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3888007870 tx=0x7f388800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 shutdown_connections 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6e40aa0 0x5636f6e41f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e31060 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6ca7df0 0x5636f6e305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2710865533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6d939a0 0x5636f6e30b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 >> 172.21.15.16:0/2710865533 conn(0x5636f6d41340 msgr2=0x5636f6e262c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f3877400700 0 client.15273 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:00:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 shutdown_connections 2024-05-27T07:00:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2710865533 wait complete. 2024-05-27T07:00:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.131+0000 7f38a6a07b80 20 client.15273 trim_cache size 0 max 16384 2024-05-27T07:00:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.155+0000 7f38a6a07b80 1 Processor -- start 2024-05-27T07:00:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.155+0000 7f38a6a07b80 1 -- start start 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.155+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e2be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.155+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.155+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e40250 con 0x5636f6e26e20 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e40920 con 0x5636f6ca7df0 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e40ff0 con 0x5636f6d939a0 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f3895e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f3895e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42598/0 (socket says 172.21.15.16:42598) 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f3895e00700 1 -- 172.21.15.16:0/2849272034 learned_addr learned my addr 172.21.15.16:0/2849272034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f3895400700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e2be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.156+0000 7f3894a00700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 -- 172.21.15.16:0/2849272034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 msgr2=0x5636f6e3ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e3ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 -- 172.21.15.16:0/2849272034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 msgr2=0x5636f6e2be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e2be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 -- 172.21.15.16:0/2849272034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f6e2eb50 con 0x5636f6e26e20 2024-05-27T07:00:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895400700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e2be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3895e00700 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3888009e60 tx=0x7f3888009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eac7914dfde137 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3882000700 1 -- 172.21.15.16:0/2849272034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888032070 con 0x5636f6e26e20 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3882000700 1 -- 172.21.15.16:0/2849272034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3888007b80 con 0x5636f6e26e20 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.157+0000 7f3882000700 1 -- 172.21.15.16:0/2849272034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388801f800 con 0x5636f6e26e20 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2849272034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 msgr2=0x5636f6e224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3888009e60 tx=0x7f3888009780 comp rx=0 tx=0).stop 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2849272034 shutdown_connections 2024-05-27T07:00:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6e26e20 0x5636f6e224a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e2be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2849272034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6d939a0 0x5636f6e3ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.158+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2849272034 >> 172.21.15.16:0/2849272034 conn(0x5636f6d41340 msgr2=0x5636f6e305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.159+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2849272034 shutdown_connections 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.159+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2849272034 wait complete. 2024-05-27T07:00:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.159+0000 7f38a6a07b80 1 -- start start 2024-05-27T07:00:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.159+0000 7f38a6a07b80 1 Processor -- start 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e4bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 0x5636f6e48e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e41d90 con 0x5636f6d939a0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e416c0 con 0x5636f6ca7df0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5636f6e42460 con 0x5636f6e26e20 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f38a6a07b80 10 client.0 _ll_register_callbacks cb 0x5636f6ceec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f3894a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f3894a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42608/0 (socket says 172.21.15.16:42608) 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f3894a00700 1 -- 172.21.15.16:0/2013884042 learned_addr learned my addr 172.21.15.16:0/2013884042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.160+0000 7f3895400700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e4bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3895e00700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 0x5636f6e48e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 -- 172.21.15.16:0/2013884042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 msgr2=0x5636f6e48e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 0x5636f6e48e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 -- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 msgr2=0x5636f6e4bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e4bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 -- 172.21.15.16:0/2013884042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f6e49650 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3895e00700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 0x5636f6e48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3894a00700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f387c0a52b0 tx=0x7f387c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e6406285836c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3877400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c018480 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.161+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f387c0ad1b0 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3888007b80 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f387c0a9280 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 10 client.15279 fetch_fsmap learned FSMap version 30 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636f6eac230 con 0x5636f6d939a0 2024-05-27T07:00:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3888007b80 con 0x5636f6d939a0 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f387c0a94e0 con 0x5636f6d939a0 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 20 client.15279 populate_metadata read hostname 'smithi016' 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.162+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6eae6f0 0x5636f6eafb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.163+0000 7f38a6a07b80 4 client.15279 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:00:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.163+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "status"}) -- 0x5636f6d50520 con 0x5636f6eae6f0 2024-05-27T07:00:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.163+0000 7f3895400700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6eae6f0 0x5636f6eafb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.164+0000 7f3895400700 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6eae6f0 0x5636f6eafb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e78b289b276022 server_cookie=974cb1b7fb213cd in_seq=0 out_seq=0 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.164+0000 7f3877400700 10 client.15279 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f3877400700 1 -- 172.21.15.16:0/2013884042 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5636f6d50520 con 0x5636f6eae6f0 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f3877400700 10 client.15279 handle_command_reply: tid=0 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 client.15279 shutdown 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 msgr2=0x5636f6e488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f387c0a52b0 tx=0x7f387c0077b0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 shutdown_connections 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5636f6eae6f0 0x5636f6eafb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636f6e26e20 0x5636f6e48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636f6ca7df0 0x5636f6e4bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 --2- 172.21.15.16:0/2013884042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d939a0 0x5636f6e488e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 >> 172.21.15.16:0/2013884042 conn(0x5636f6d41340 msgr2=0x5636f6e27e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f3877400700 0 client.15279 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 shutdown_connections 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.165+0000 7f38a6a07b80 1 -- 172.21.15.16:0/2013884042 wait complete. 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f38a6a07b80 20 client.15279 trim_cache size 0 max 16384 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "3c4673a4-1bf4-11ef-bc9b-c7b262605968", 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34517, 2024-05-27T07:00:41.160 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 326.77083539300003, 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 90, 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 352.407795158 2024-05-27T07:00:41.161 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 msgr2=0x7f38780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 0x7f38780b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5636f6e28fa0 tx=0x7f38880099e0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 msgr2=0x7f389019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d980 tx=0x7f387c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 shutdown_connections 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f38780b2900 0x7f38780b4e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3890110cc0 0x7f38901a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.166+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3890103110 0x7f389019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.167+0000 7f3897e00700 1 --2- 172.21.15.16:0/286906206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3890103a60 0x7f389019d420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.167+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 >> 172.21.15.16:0/286906206 conn(0x7f38900fee40 msgr2=0x7f3890101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.167+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 shutdown_connections 2024-05-27T07:00:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:41.167+0000 7f3897e00700 1 -- 172.21.15.16:0/286906206 wait complete. 2024-05-27T07:00:41.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:41 smithi164 ceph-mon[35327]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:41 smithi016 ceph-mon[31415]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:41.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:41 smithi069 ceph-mon[35276]: pgmap v561: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:00:42.256 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T07:00:42.266 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T07:00:42.266 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T07:00:42.266 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T07:00:42.266 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-27T07:00:42.267 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:00:42.267 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:00:42.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-27T07:00:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 -- 172.21.15.16:0/3408258376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 msgr2=0x7f255410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 --2- 172.21.15.16:0/3408258376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f255410ae20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f254402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 -- 172.21.15.16:0/3408258376 shutdown_connections 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 --2- 172.21.15.16:0/3408258376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 0x7f2554112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 --2- 172.21.15.16:0/3408258376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 0x7f255410b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 --2- 172.21.15.16:0/3408258376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f255410ae20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 -- 172.21.15.16:0/3408258376 >> 172.21.15.16:0/3408258376 conn(0x7f2554078d20 msgr2=0x7f255407b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.566+0000 7f2559000640 1 -- 172.21.15.16:0/3408258376 shutdown_connections 2024-05-27T07:00:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.567+0000 7f2559000640 1 -- 172.21.15.16:0/3408258376 wait complete. 2024-05-27T07:00:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 Processor -- start 2024-05-27T07:00:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 -- start start 2024-05-27T07:00:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 0x7f25541a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 0x7f25541a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2554115aa0 con 0x7f255410a3f0 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2554115920 con 0x7f255410b360 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.568+0000 7f2559000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2554115c20 con 0x7f2554110060 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.569+0000 7f2553400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.569+0000 7f2553400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47500/0 (socket says 172.21.15.16:47500) 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.569+0000 7f2553400640 1 -- 172.21.15.16:0/2168404775 learned_addr learned my addr 172.21.15.16:0/2168404775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.569+0000 7f254aa00640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 0x7f25541a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.569+0000 7f2553e00640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 0x7f25541a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 -- 172.21.15.16:0/2168404775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 msgr2=0x7f25541a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 0x7f25541a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 -- 172.21.15.16:0/2168404775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 msgr2=0x7f25541a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 0x7f25541a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 -- 172.21.15.16:0/2168404775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25541a8200 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2553400640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2544009430 tx=0x7f2544003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544004040 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2544004670 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25541a8490 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.570+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544030d60 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.571+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25541a89e0 con 0x7f255410a3f0 2024-05-27T07:00:42.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.572+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f255410ee70 con 0x7f255410a3f0 2024-05-27T07:00:42.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.575+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f25440305c0 con 0x7f255410a3f0 2024-05-27T07:00:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.577+0000 7f2550c00640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 0x7f25380b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.578+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2544103ce0 con 0x7f255410a3f0 2024-05-27T07:00:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.578+0000 7f254aa00640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 0x7f25380b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.578+0000 7f254aa00640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 0x7f25380b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25541a1760 tx=0x7f253c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:42.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.581+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2544004820 con 0x7f255410a3f0 2024-05-27T07:00:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:42.849+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f25540638c0 con 0x7f255410a3f0 2024-05-27T07:00:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.420+0000 7f2550c00640 1 -- 172.21.15.16:0/2168404775 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f25440cb6e0 con 0x7f255410a3f0 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 msgr2=0x7f25380b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 0x7f25380b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25541a1760 tx=0x7f253c00b390 comp rx=0 tx=0).stop 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 msgr2=0x7f25541a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2544009430 tx=0x7f2544003a70 comp rx=0 tx=0).stop 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 shutdown_connections 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f25380b27e0 0x7f25380b4cf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:43.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2554110060 0x7f25541a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:43.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f255410b360 0x7f25541a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:43.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 --2- 172.21.15.16:0/2168404775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f255410a3f0 0x7f25541a06c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:43.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 >> 172.21.15.16:0/2168404775 conn(0x7f2554078d20 msgr2=0x7f255410c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:43.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.424+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 shutdown_connections 2024-05-27T07:00:43.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:43.425+0000 7f2559000640 1 -- 172.21.15.16:0/2168404775 wait complete. 2024-05-27T07:00:43.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:43 smithi016 ceph-mon[31415]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:43.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:00:43.463 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-27T07:00:43.474 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-27T07:00:43.474 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-27T07:00:43.474 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-27T07:00:43.474 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T07:00:43.474 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:43.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:43.475 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-27T07:00:43.522 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-27T07:00:43.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:43.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:00:43.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:43.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-05-27T07:00:43.733 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setuser ceph since I am not root 2024-05-27T07:00:43.734 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: ignoring --setgroup ceph since I am not root 2024-05-27T07:00:43.757 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-27T07:00:43.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:43 smithi069 ceph-mon[35276]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: ignoring --setuser ceph since I am not root 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: ignoring --setgroup ceph since I am not root 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:43 smithi069 ceph-mgr[36644]: -- 172.21.15.69:0/1454288539 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56290f8da4e0 con 0x562914666900 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:43 smithi069 ceph-mgr[36644]: -- 172.21.15.69:0/1454288539 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56290f8dc5a0 con 0x562914666900 2024-05-27T07:00:43.828 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:43.835+0000 7fe73ba3b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:00:43.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:43 smithi164 ceph-mon[35327]: pgmap v562: 129 pgs: 129 active+clean; 650 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:00:43.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:00:43.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: ignoring --setuser ceph since I am not root 2024-05-27T07:00:43.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: ignoring --setgroup ceph since I am not root 2024-05-27T07:00:43.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:43 smithi164 ceph-mgr[37168]: -- 172.21.15.164:0/2083250741 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2ef9865a0 con 0x55c2f4710900 2024-05-27T07:00:44.109 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:43.882+0000 7f2af8d81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:00:44.125 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:43.872+0000 7f5911e92200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:00:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:44.368 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:44.089+0000 7fe73ba3b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:44.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:44.229+0000 7fe73ba3b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:00:44.445 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:44.164+0000 7f2af8d81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:44.445 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:44.333+0000 7f2af8d81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:00:44.446 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:44.133+0000 7f5911e92200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:44.446 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:44.272+0000 7f5911e92200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:00:44.720 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:00:44.720 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:44 smithi016 ceph-mon[31415]: mgrmap e19: x(active, since 16m), standbys: z, y 2024-05-27T07:00:44.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:00:44.734 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:44 smithi069 ceph-mon[35276]: mgrmap e19: x(active, since 16m), standbys: z, y 2024-05-27T07:00:44.734 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:44.628+0000 7fe73ba3b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:00:44.781 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2168404775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:00:44.782 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:44 smithi164 ceph-mon[35327]: mgrmap e19: x(active, since 16m), standbys: z, y 2024-05-27T07:00:44.782 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:44.674+0000 7f5911e92200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:00:45.082 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:44.789+0000 7f5911e92200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:00:45.109 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:44.758+0000 7f2af8d81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:00:45.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:44.881+0000 7f2af8d81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:00:45.118 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:44.742+0000 7fe73ba3b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:00:46.619 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:46.286+0000 7fe73ba3b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:00:46.619 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:46.530+0000 7fe73ba3b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.751+0000 7f5ebae00700 1 -- 172.21.15.16:0/2603656675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 msgr2=0x7f5eb4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.751+0000 7f5ebae00700 1 --2- 172.21.15.16:0/2603656675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb4076130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009990 tx=0x7f5ea4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 -- 172.21.15.16:0/2603656675 shutdown_connections 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 --2- 172.21.15.16:0/2603656675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 0x7f5eb4112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 --2- 172.21.15.16:0/2603656675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb4076130 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 --2- 172.21.15.16:0/2603656675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 0x7f5eb4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 -- 172.21.15.16:0/2603656675 >> 172.21.15.16:0/2603656675 conn(0x7f5eb4100030 msgr2=0x7f5eb4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 -- 172.21.15.16:0/2603656675 shutdown_connections 2024-05-27T07:00:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.752+0000 7f5ebae00700 1 -- 172.21.15.16:0/2603656675 wait complete. 2024-05-27T07:00:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.753+0000 7f5ebae00700 1 Processor -- start 2024-05-27T07:00:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.753+0000 7f5ebae00700 1 -- start start 2024-05-27T07:00:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 0x7f5eb419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 0x7f5eb41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4116530 con 0x7f5eb4075d20 2024-05-27T07:00:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41163b0 con 0x7f5eb4110b90 2024-05-27T07:00:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.754+0000 7f5ebae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41166b0 con 0x7f5eb4076670 2024-05-27T07:00:46.747 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:46.549+0000 7f2af8d81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47532/0 (socket says 172.21.15.16:47532) 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 -- 172.21.15.16:0/1045750183 learned_addr learned my addr 172.21.15.16:0/1045750183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3400700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 0x7f5eb419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb8e00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 0x7f5eb41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 msgr2=0x7f5eb419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 0x7f5eb419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 msgr2=0x7f5eb41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 0x7f5eb41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.755+0000 7f5eb3e00700 1 -- 172.21.15.16:0/1045750183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb41a1dd0 con 0x7f5eb4075d20 2024-05-27T07:00:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.756+0000 7f5eb3e00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40057c0 tx=0x7f5ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.756+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4027070 con 0x7f5eb4075d20 2024-05-27T07:00:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.756+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ea400fb00 con 0x7f5eb4075d20 2024-05-27T07:00:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.756+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4022ad0 con 0x7f5eb4075d20 2024-05-27T07:00:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.756+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb41a2060 con 0x7f5eb4075d20 2024-05-27T07:00:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.757+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41a24f0 con 0x7f5eb4075d20 2024-05-27T07:00:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.758+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb40512b0 con 0x7f5eb4075d20 2024-05-27T07:00:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.761+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f5ea402a070 con 0x7f5eb4075d20 2024-05-27T07:00:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.764+0000 7f5eb0c00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 0x7f5e980b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.764+0000 7f5eb3400700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 msgr2=0x7f5e980b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.764+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5ea40e38d0 con 0x7f5eb4075d20 2024-05-27T07:00:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.764+0000 7f5eb3400700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 0x7f5e980b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:00:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.766+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ea4013070 con 0x7f5eb4075d20 2024-05-27T07:00:46.795 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:46.344+0000 7f5911e92200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:00:46.795 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:46.582+0000 7f5911e92200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:46.796 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:46.803+0000 7f5911e92200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:00:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.965+0000 7f5eb3400700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 msgr2=0x7f5e980b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:46.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:46.965+0000 7f5eb3400700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 0x7f5e980b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:00:47.031 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:46.813+0000 7f2af8d81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:47.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.049+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5eb402e5e0 con 0x7f5eb4075d20 2024-05-27T07:00:47.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.050+0000 7f5eb0c00700 1 -- 172.21.15.16:0/1045750183 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+88 (secure 0 0 0) 0x7f5ea40ab270 con 0x7f5eb4075d20 2024-05-27T07:00:47.042 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T07:00:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 msgr2=0x7f5e980b4de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:00:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 0x7f5e980b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 msgr2=0x7f5eb419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40057c0 tx=0x7f5ea40057f0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 shutdown_connections 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f5e980b28d0 0x7f5e980b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5eb4110b90 0x7f5eb41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb4075d20 0x7f5eb419cf00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 --2- 172.21.15.16:0/1045750183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb4076670 0x7f5eb419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 >> 172.21.15.16:0/1045750183 conn(0x7f5eb4100030 msgr2=0x7f5eb4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 shutdown_connections 2024-05-27T07:00:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:47.053+0000 7f5ebae00700 1 -- 172.21.15.16:0/1045750183 wait complete. 2024-05-27T07:00:47.096 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:46.751+0000 7fe73ba3b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:00:47.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:47 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1045750183' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:00:47.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:47.038+0000 7f2af8d81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:00:47.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:47 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1045750183' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:00:47.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:47.138+0000 7fe73ba3b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:00:47.369 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:47.264+0000 7fe73ba3b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:00:47.568 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:47 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1045750183' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:00:47.569 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:47.192+0000 7f5911e92200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:00:47.569 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:47.319+0000 7f5911e92200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:00:47.769 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:47.518+0000 7fe73ba3b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:00:47.828 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:47.576+0000 7f5911e92200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:00:47.841 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:47.453+0000 7f2af8d81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:00:47.841 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:47.583+0000 7f2af8d81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T07:00:48.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T07:00:48.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T07:00:48.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T07:00:48.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:48.083 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-05-27T07:00:48.084 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:47.836+0000 7f5911e92200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:48.109 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:47.848+0000 7f2af8d81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:00:48.118 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:47.777+0000 7fe73ba3b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:48.132 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T07:00:48.132 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T07:00:48.132 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:00:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-05-27T07:00:48.134 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T07:00:48.134 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:00:48.134 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-05-27T07:00:48.134 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41483sec preferred_lft 41483sec 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:00:48.135 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-05-27T07:00:48.136 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T07:00:48.136 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:00:48.136 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T07:00:48.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-05-27T07:00:48.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-05-27T07:00:48.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-05-27T07:00:48.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T07:00:48.138 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T07:00:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:00:48.376 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:48.129+0000 7f2af8d81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:00:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:48.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:48.417 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T07:00:48.478 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-05-27T07:00:48.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:48.483 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-05-27T07:00:48.533 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-05-27T07:00:48.533 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T07:00:48.533 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-27T07:00:48.533 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-27T07:00:48.534 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-27T07:00:48.535 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T07:00:48.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:00:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:48.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:48.732 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-27T07:00:48.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:48.780 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-05-27T07:00:48.821 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:00:48.821 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T07:00:48.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:48.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T07:00:48.821 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T07:00:48.857 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:48.572+0000 7fe73ba3b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:00:48.931 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:48.644+0000 7f5911e92200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:00:48.931 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:48.938+0000 7f5911e92200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:00:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:00:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:49.045 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T07:00:49.045 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:00:49.045 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T07:00:49.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T07:00:49.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T07:00:49.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T07:00:49.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T07:00:49.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T07:00:49.046 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T07:00:49.118 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:48.866+0000 7fe73ba3b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:00:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:00:49.244 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:48.990+0000 7f2af8d81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:00:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:49.293 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:00:49.293 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-27T07:00:49.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-05-27T07:00:49.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T07:00:49.293 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-27T07:00:49.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:00:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:49.484 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-27T07:00:49.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:00:49.484 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:49.524 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T07:00:49.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:00:49.524 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:49.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T07:00:49.609 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:49.310+0000 7f2af8d81200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:00:49.690 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T07:00:49.731 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:00:49.732 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T07:00:49.733 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T07:00:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T07:00:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T07:00:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T07:00:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:49.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:49.735 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T07:00:49.820 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T07:00:49.820 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/c2880afe-8509-41f8-835f-a35eb3c9cbaf --client_fs=cephfs) 2024-05-27T07:00:49.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:49.506+0000 7f5911e92200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:00:49.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:49.637+0000 7f5911e92200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:00:49.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T07:00:49.868 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:49.427+0000 7fe73ba3b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:00:49.868 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:49.557+0000 7fe73ba3b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:00:49.937 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T07:00:49.983 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T07:00:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T07:00:49.985 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:49.986 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:49.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:49.987 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T07:00:50.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-05-27T07:00:50.138+0000 7fd504a35480 -1 init, newargv = 0x7fd4f00046a0 newargc=15 2024-05-27T07:00:50.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[89660]: starting ceph client 2024-05-27T07:00:50.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[89660]: starting fuse 2024-05-27T07:00:50.359 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:49.938+0000 7f2af8d81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:00:50.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:50.101+0000 7f2af8d81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:00:50.368 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:50.027+0000 7fe73ba3b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:00:50.566 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:50.113+0000 7f5911e92200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:00:50.567 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:50.574+0000 7f5911e92200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:00:50.832 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:50.702+0000 7f5911e92200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:00:50.859 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:50.604+0000 7f2af8d81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:00:50.868 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:50.494+0000 7fe73ba3b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:00:50.868 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:50.621+0000 7fe73ba3b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:00:51.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-27T07:00:51.249 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:50.946+0000 7fe73ba3b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:00:51.249 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:51.126+0000 7fe73ba3b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:00:51.252 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:00:51.301 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-05-27T07:00:51.302 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:00:51.303 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/61a892a403ad19726b8c4d2b743b08ef866b259957526483e3d5945214988909/merged 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e5e6cbfe1878fcd7a14a68a29b817f14f98cfc12d5e713f1a60fb8098a48a772/merged 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/f2877e829c2a4991daed65b66671ede410d1046a7fab6372d680bf69f0ef30f5/merged 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a5bd921b688acbe4d078a7e4b876eca9524b2b9445c43507df6e832ebffc1c3c/merged 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/79842ef4eca9004bfebfcdce91696c56bdce5ddcee323c30b78143cf57666eca/merged 2024-05-27T07:00:51.304 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/50523f31cf9822995e7f3713e13fa1dc86d271641dd3f22331fc49ca21086100/merged 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e6198dfcfead6e2ce392181588d7912445fd7b437c3224f6ec0b7f6f4457db27/merged 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/224b47adefb8e09fbf85bcb28c52c947a9a3909e67717420a03e9d465c15f67b/merged 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/101a10472684d5c989f201f267e11435652b1195ae45c3948eaa566f52446af4/merged 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/fd60a753bbad228f9b779e2ff9c7aac6e8dc5c3b6a94d997c7c6be4be53d62d9/merged 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:00:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:51.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:51.306 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-27T07:00:51.332 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:51.031+0000 7f5911e92200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:00:51.332 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:51.211+0000 7f5911e92200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:00:51.351 INFO:teuthology.orchestra.run.smithi016.stdout:119 2024-05-27T07:00:51.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-27T07:00:51.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-05-27T07:00:51.352 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-05-27T07:00:51.353 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-05-27T07:00:51.359 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:51.094+0000 7f2af8d81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:00:51.428 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:51.230+0000 7f2af8d81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:00:51.598 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.89660.asok 2024-05-27T07:00:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:00:51.614 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89660.asok 2024-05-27T07:00:51.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:51.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89660.asok status 2024-05-27T07:00:51.627 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:51.257+0000 7fe73ba3b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:00:51.748 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:51.345+0000 7f5911e92200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:00:51.748 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:51.755+0000 7f5911e92200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:51.769 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:51.579+0000 7f2af8d81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:00:51.769 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:51.775+0000 7f2af8d81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-05-27T07:00:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "89660", 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/volumes/qa/sv_0/c2880afe-8509-41f8-835f-a35eb3c9cbaf" 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 15315, 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 15315 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-27T07:00:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2039046124 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2039046124 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.15315 192.168.0.1:0/2039046124", 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2039046124", 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 5, 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 30, 2024-05-27T07:00:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 90, 2024-05-27T07:00:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-05-27T07:00:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-05-27T07:00:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-05-27T07:00:51.927 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:00:51.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:51.946 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-05-27T07:00:51.946 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:52.004 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-05-27T07:00:52.004 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:52.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:00:52.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T07:00:52.082 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:51.881+0000 7f5911e92200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:00:52.091 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:51.919+0000 7f2af8d81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:00:52.108 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:00:52.128 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:00:52.128 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:00:52.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-27T07:00:52.129 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:51.661+0000 7fe73ba3b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:52.129 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:51.787+0000 7fe73ba3b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:00:52.359 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:52.344+0000 7f2af8d81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 -- 172.21.15.16:0/449390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c1099c0 msgr2=0x7f1f9c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 --2- 172.21.15.16:0/449390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c10bde0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009ca0 tx=0x7f1f9402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 -- 172.21.15.16:0/449390390 shutdown_connections 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 --2- 172.21.15.16:0/449390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c10bde0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 --2- 172.21.15.16:0/449390390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 --2- 172.21.15.16:0/449390390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c103a80 0x7f1f9c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 -- 172.21.15.16:0/449390390 >> 172.21.15.16:0/449390390 conn(0x7f1f9c0fec70 msgr2=0x7f1f9c101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.408+0000 7f1fa1600640 1 -- 172.21.15.16:0/449390390 shutdown_connections 2024-05-27T07:00:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.409+0000 7f1fa1600640 1 -- 172.21.15.16:0/449390390 wait complete. 2024-05-27T07:00:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 Processor -- start 2024-05-27T07:00:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 -- start start 2024-05-27T07:00:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 0x7f1f9c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f4b0 con 0x7f1f9c104cc0 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f330 con 0x7f1f9c1099c0 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1fa1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f630 con 0x7f1f9c103a80 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.410+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58622/0 (socket says 172.21.15.16:58622) 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 -- 172.21.15.16:0/1346158155 learned_addr learned my addr 172.21.15.16:0/1346158155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9be00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 0x7f1f9c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1fa0c00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 msgr2=0x7f1f9c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 0x7f1f9c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1099c0 msgr2=0x7f1f9c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 -- 172.21.15.16:0/1346158155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1a1b90 con 0x7f1f9c104cc0 2024-05-27T07:00:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9be00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 0x7f1f9c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:00:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.411+0000 7f1f9b400640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d770 tx=0x7f1f8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.412+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88017070 con 0x7f1f9c104cc0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.412+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f880054b0 con 0x7f1f9c104cc0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.412+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004c60 con 0x7f1f9c104cc0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.412+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1a1e80 con 0x7f1f9c104cc0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.412+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a22f0 con 0x7f1f9c104cc0 2024-05-27T07:00:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.413+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c1087d0 con 0x7f1f9c104cc0 2024-05-27T07:00:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.416+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f1f88004e00 con 0x7f1f9c104cc0 2024-05-27T07:00:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.418+0000 7f1f98c00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 0x7f1f800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.418+0000 7f1f9be00640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 msgr2=0x7f1f800b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.418+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f880e20e0 con 0x7f1f9c104cc0 2024-05-27T07:00:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.418+0000 7f1f9be00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 0x7f1f800b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:00:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.423+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f880a9a50 con 0x7f1f9c104cc0 2024-05-27T07:00:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.619+0000 7f1f9be00640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 msgr2=0x7f1f800b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.619+0000 7f1f9be00640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 0x7f1f800b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:00:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.676+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f1f9c02dc20 con 0x7f1f9c104cc0 2024-05-27T07:00:52.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.731+0000 7f1f98c00640 1 -- 172.21.15.16:0/1346158155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f1f880a9350 con 0x7f1f9c104cc0 2024-05-27T07:00:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.734+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 msgr2=0x7f1f800b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:00:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.734+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 0x7f1f800b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 msgr2=0x7f1f9c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d770 tx=0x7f1f8800dc30 comp rx=0 tx=0).stop 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 shutdown_connections 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f1f800b28b0 0x7f1f800b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1099c0 0x7f1f9c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104cc0 0x7f1f9c19a590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 --2- 172.21.15.16:0/1346158155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f9c103a80 0x7f1f9c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 >> 172.21.15.16:0/1346158155 conn(0x7f1f9c0fec70 msgr2=0x7f1f9c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 shutdown_connections 2024-05-27T07:00:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:52.735+0000 7f1fa1600640 1 -- 172.21.15.16:0/1346158155 wait complete. 2024-05-27T07:00:52.734 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:52.479+0000 7f2af8d81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:00:52.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-27T07:00:52.844 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:52.557+0000 7fe73ba3b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:00:52.845 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:52 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:00:52.936 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:52 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:00:52.936 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:52.652+0000 7f5911e92200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:00:52.936 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:00:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z[37164]: 2024-05-27T07:00:52.943+0000 7f5911e92200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.027+0000 7f7fbf200640 1 -- 172.21.15.16:0/408484153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8103c60 msgr2=0x7f7fb8104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.027+0000 7f7fbf200640 1 --2- 172.21.15.16:0/408484153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8103c60 0x7f7fb8104960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009900 tx=0x7f7fa802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 -- 172.21.15.16:0/408484153 shutdown_connections 2024-05-27T07:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 --2- 172.21.15.16:0/408484153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb8109ba0 0x7f7fb810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 --2- 172.21.15.16:0/408484153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104ea0 0x7f7fb81052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 --2- 172.21.15.16:0/408484153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8103c60 0x7f7fb8104960 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 -- 172.21.15.16:0/408484153 >> 172.21.15.16:0/408484153 conn(0x7f7fb80fee50 msgr2=0x7f7fb8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 -- 172.21.15.16:0/408484153 shutdown_connections 2024-05-27T07:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.028+0000 7f7fbf200640 1 -- 172.21.15.16:0/408484153 wait complete. 2024-05-27T07:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 Processor -- start 2024-05-27T07:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 -- start start 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8103c60 0x7f7fb819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 0x7f7fb819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb8109ba0 0x7f7fb81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810f660 con 0x7f7fb8104ea0 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810f4e0 con 0x7f7fb8109ba0 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.029+0000 7f7fbf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810f7e0 con 0x7f7fb8103c60 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 0x7f7fb819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 0x7f7fb819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58632/0 (socket says 172.21.15.16:58632) 2024-05-27T07:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 -- 172.21.15.16:0/3281182375 learned_addr learned my addr 172.21.15.16:0/3281182375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8103c60 0x7f7fb819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fbd200640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb8109ba0 0x7f7fb81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8103c60 msgr2=0x7f7fb819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8103c60 0x7f7fb819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb8109ba0 msgr2=0x7f7fb81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb8109ba0 0x7f7fb81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.030+0000 7f7fafe00640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb81a1d20 con 0x7f7fb8104ea0 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fafe00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 0x7f7fb819a7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e9f0 tx=0x7f7fa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000cd80 con 0x7f7fb8104ea0 2024-05-27T07:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fa00054b0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb81a1fb0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0010430 con 0x7f7fb8104ea0 2024-05-27T07:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.031+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb81a2420 con 0x7f7fb8104ea0 2024-05-27T07:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.032+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb81089b0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.035+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f7fa00046c0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.036+0000 7f7fb5600640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 0x7f7f9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.037+0000 7f7fb7e00640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 msgr2=0x7f7f9c0b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.037+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7fa00e2ca0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.037+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 0x7f7f9c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:00:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.039+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fa0054df0 con 0x7f7fb8104ea0 2024-05-27T07:00:53.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:52 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:00:53.118 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:00:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y[36640]: 2024-05-27T07:00:52.852+0000 7fe73ba3b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:00:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.238+0000 7f7fb7e00640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 msgr2=0x7f7f9c0b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.238+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 0x7f7f9c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:00:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.277+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7fb80631a0 con 0x7f7f9c0b2830 2024-05-27T07:00:53.609 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:53.310+0000 7f2af8d81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:00:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.639+0000 7f7fb7e00640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 msgr2=0x7f7f9c0b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2755801148 2024-05-27T07:00:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.639+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 0x7f7f9c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:00:53.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.654+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7f7fa00a6140 con 0x7f7fb8104ea0 2024-05-27T07:00:53.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.655+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 msgr2=0x7f7f9c0b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:00:53.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:53.655+0000 7f7fb5600640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/2755801148,v1:172.21.15.16:6801/2755801148] conn(0x7f7f9c0b2830 0x7f7f9c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:53.948 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:53.619+0000 7f2af8d81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Standby manager daemon y restarted 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Standby manager daemon y started 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Standby manager daemon z restarted 2024-05-27T07:00:53.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Standby manager daemon z started 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Active manager daemon x restarted 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Activating manager daemon x 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: mgrmap e20: x(active, starting, since 0.0295992s), standbys: z, y 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:00:53.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:00:53.951 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:00:53.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:00:53.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:00:53.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:00:53.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:00:53.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:53 smithi016 ceph-mon[31415]: Manager daemon x is now available 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Standby manager daemon y restarted 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Standby manager daemon y started 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:00:53.962 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Standby manager daemon z restarted 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Standby manager daemon z started 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Active manager daemon x restarted 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Activating manager daemon x 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: mgrmap e20: x(active, starting, since 0.0295992s), standbys: z, y 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:00:53.963 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:00:53.964 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:00:53.965 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:00:53.966 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:53 smithi069 ceph-mon[35276]: Manager daemon x is now available 2024-05-27T07:00:54.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1346158155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:00:54.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:00:54.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Standby manager daemon y restarted 2024-05-27T07:00:54.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Standby manager daemon y started 2024-05-27T07:00:54.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.69:0/3739970155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Standby manager daemon z restarted 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Standby manager daemon z started 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.? 172.21.15.164:0/69292611' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Active manager daemon x restarted 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Activating manager daemon x 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:00:54.002 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: mgrmap e20: x(active, starting, since 0.0295992s), standbys: z, y 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:00:54.003 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:00:54.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:00:54.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:00:54.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:53 smithi164 ceph-mon[35327]: Manager daemon x is now available 2024-05-27T07:00:54.198 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.186+0000 7f2a4d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.198 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.186+0000 7f2a4d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.198 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.186+0000 7f2a4d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.186+0000 7f2a4d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.186+0000 7f2a4d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.187+0000 7f2a4a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.187+0000 7f2a4a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.187+0000 7f2a4a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.187+0000 7f2a4a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.187+0000 7f2a4a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.675+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 21) ==== 149464+0+0 (secure 0 0 0) 0x7f7fa00e0410 con 0x7f7fb8104ea0 2024-05-27T07:00:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.676+0000 7f7fb5600640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7f9c075070 0x7f7f9c075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.676+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7f9c042310 con 0x7f7f9c075070 2024-05-27T07:00:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.683+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7f9c075070 0x7f7f9c075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.689+0000 7f7fb7e00640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7f9c075070 0x7f7f9c075480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fa802a910 tx=0x7f7fa8031400 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:54.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.743+0000 7f7fb5600640 1 -- 172.21.15.16:0/3281182375 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7f9c042310 con 0x7f7f9c075070 2024-05-27T07:00:54.735 INFO:teuthology.orchestra.run.smithi016.stdout:{} 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.745+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7f9c075070 msgr2=0x7f7f9c075480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.745+0000 7f7fbf200640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7f9c075070 0x7f7f9c075480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fa802a910 tx=0x7f7fa8031400 comp rx=0 tx=0).stop 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.745+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 msgr2=0x7f7fb819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.745+0000 7f7fbf200640 1 --2- 172.21.15.16:0/3281182375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8104ea0 0x7f7fb819a7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e9f0 tx=0x7f7fa000edf0 comp rx=0 tx=0).stop 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.745+0000 7f7fbd200640 1 -- 172.21.15.16:0/3281182375 reap_dead start 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.746+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 shutdown_connections 2024-05-27T07:00:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.746+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 >> 172.21.15.16:0/3281182375 conn(0x7f7fb80fee50 msgr2=0x7f7fb8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.746+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 shutdown_connections 2024-05-27T07:00:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:54.746+0000 7f7fbf200640 1 -- 172.21.15.16:0/3281182375 wait complete. 2024-05-27T07:00:54.770 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-27T07:00:54.781 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-27T07:00:54.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: [27/May/2024:07:00:54] ENGINE Bus STARTING 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: [27/May/2024:07:00:54] ENGINE Serving on https://172.21.15.16:7150 2024-05-27T07:00:55.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: [27/May/2024:07:00:54] ENGINE Serving on http://172.21.15.16:8765 2024-05-27T07:00:55.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: [27/May/2024:07:00:54] ENGINE Bus STARTED 2024-05-27T07:00:55.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:54 smithi164 ceph-mon[35327]: mgrmap e21: x(active, since 1.05895s), standbys: z, y 2024-05-27T07:00:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 -- 172.21.15.16:0/2266690808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00109b60 msgr2=0x7fbe0010bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 --2- 172.21.15.16:0/2266690808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00109b60 0x7fbe0010bf80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009ca0 tx=0x7fbdf802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 -- 172.21.15.16:0/2266690808 shutdown_connections 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 --2- 172.21.15.16:0/2266690808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00109b60 0x7fbe0010bf80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 --2- 172.21.15.16:0/2266690808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00104e60 0x7fbe00105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 --2- 172.21.15.16:0/2266690808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00103c20 0x7fbe00104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 -- 172.21.15.16:0/2266690808 >> 172.21.15.16:0/2266690808 conn(0x7fbe000fee30 msgr2=0x7fbe001012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 -- 172.21.15.16:0/2266690808 shutdown_connections 2024-05-27T07:00:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.091+0000 7fbe06c00640 1 -- 172.21.15.16:0/2266690808 wait complete. 2024-05-27T07:00:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.092+0000 7fbe06c00640 1 Processor -- start 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.092+0000 7fbe06c00640 1 -- start start 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.092+0000 7fbe06c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe06c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 0x7fbe0019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe06c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00109b60 0x7fbe001a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe06c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0010f650 con 0x7fbe00104e60 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe06c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0010f4d0 con 0x7fbe00109b60 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe06c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0010f7d0 con 0x7fbe00103c20 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 0x7fbe0019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44654/0 (socket says 172.21.15.16:44654) 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 -- 172.21.15.16:0/4259966872 learned_addr learned my addr 172.21.15.16:0/4259966872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbe04c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00109b60 0x7fbe001a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 -- 172.21.15.16:0/4259966872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00109b60 msgr2=0x7fbe001a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00109b60 0x7fbe001a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.093+0000 7fbdffe00640 1 -- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 msgr2=0x7fbe0019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdffe00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 0x7fbe0019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdffe00640 1 -- 172.21.15.16:0/4259966872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe001a1d20 con 0x7fbe00103c20 2024-05-27T07:00:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdff400640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 0x7fbe0019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdffe00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9f0 tx=0x7fbdf000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000ccc0 con 0x7fbe00103c20 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf00044b0 con 0x7fbe00103c20 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0010430 con 0x7fbe00103c20 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe001a1fb0 con 0x7fbe00103c20 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.094+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe001a23c0 con 0x7fbe00103c20 2024-05-27T07:00:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.095+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe00108970 con 0x7fbe00103c20 2024-05-27T07:00:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.099+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 21) ==== 149464+0+0 (secure 0 0 0) 0x7fbdf0003f90 con 0x7fbe00103c20 2024-05-27T07:00:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.100+0000 7fbdfcc00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 0x7fbde40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.101+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fbdf00e2640 con 0x7fbe00103c20 2024-05-27T07:00:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.101+0000 7fbdff400640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 0x7fbde40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.102+0000 7fbdff400640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 0x7fbde40b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe0019b2f0 tx=0x7fbdec005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.105+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdf00aa160 con 0x7fbe00103c20 2024-05-27T07:00:55.109 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.706+0000 7f2a46400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:55.109 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.706+0000 7f2a46400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.706+0000 7f2a46400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.706+0000 7f2a46400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:00:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:00:54.706+0000 7f2a46400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: [27/May/2024:07:00:54] ENGINE Bus STARTING 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: [27/May/2024:07:00:54] ENGINE Serving on https://172.21.15.16:7150 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: [27/May/2024:07:00:54] ENGINE Serving on http://172.21.15.16:8765 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: [27/May/2024:07:00:54] ENGINE Bus STARTED 2024-05-27T07:00:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:54 smithi016 ceph-mon[31415]: mgrmap e21: x(active, since 1.05895s), standbys: z, y 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: [27/May/2024:07:00:54] ENGINE Bus STARTING 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: [27/May/2024:07:00:54] ENGINE Serving on https://172.21.15.16:7150 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: [27/May/2024:07:00:54] ENGINE Serving on http://172.21.15.16:8765 2024-05-27T07:00:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: [27/May/2024:07:00:54] ENGINE Bus STARTED 2024-05-27T07:00:55.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:54 smithi069 ceph-mon[35276]: mgrmap e21: x(active, since 1.05895s), standbys: z, y 2024-05-27T07:00:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.330+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe0002dcb0 con 0x7fbe00103c20 2024-05-27T07:00:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.333+0000 7fbdfcc00640 1 -- 172.21.15.16:0/4259966872 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25084 (secure 0 0 0) 0x7fbdf00a9a60 con 0x7fbe00103c20 2024-05-27T07:00:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:00:55.326 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T07:00:53.625840+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/702672487":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6800/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/454643589":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3317326035":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3906409731":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:00:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 msgr2=0x7fbde40b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 0x7fbde40b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe0019b2f0 tx=0x7fbdec005f90 comp rx=0 tx=0).stop 2024-05-27T07:00:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 msgr2=0x7fbe0019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9f0 tx=0x7fbdf000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 shutdown_connections 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbde40b25e0 0x7fbde40b4af0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe00109b60 0x7fbe001a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe00104e60 0x7fbe0019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 --2- 172.21.15.16:0/4259966872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe00103c20 0x7fbe0019a250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 >> 172.21.15.16:0/4259966872 conn(0x7fbe000fee30 msgr2=0x7fbe00105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 shutdown_connections 2024-05-27T07:00:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.336+0000 7fbe06c00640 1 -- 172.21.15.16:0/4259966872 wait complete. 2024-05-27T07:00:55.353 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T06:47:58.074809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T06:55:03.523883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T06:55:08.339180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T06:55:24.196525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T07:00:55.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T07:00:55.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 -- 172.21.15.16:0/2107295798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f01041c0 msgr2=0x7f28f01065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 --2- 172.21.15.16:0/2107295798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f01041c0 0x7f28f01065e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009900 tx=0x7f28e002a410 comp rx=0 tx=0).stop 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 -- 172.21.15.16:0/2107295798 shutdown_connections 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 --2- 172.21.15.16:0/2107295798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 0x7f28f010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 --2- 172.21.15.16:0/2107295798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f0106bb0 0x7f28f0109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 --2- 172.21.15.16:0/2107295798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f01041c0 0x7f28f01065e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 -- 172.21.15.16:0/2107295798 >> 172.21.15.16:0/2107295798 conn(0x7f28f00fdb40 msgr2=0x7f28f00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 -- 172.21.15.16:0/2107295798 shutdown_connections 2024-05-27T07:00:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.653+0000 7f28f5a00640 1 -- 172.21.15.16:0/2107295798 wait complete. 2024-05-27T07:00:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.655+0000 7f28f5a00640 1 Processor -- start 2024-05-27T07:00:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.655+0000 7f28f5a00640 1 -- start start 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.655+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01041c0 0x7f28f0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 0x7f28f0102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010f440 con 0x7f28f0106bb0 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010f2c0 con 0x7f28f01096e0 2024-05-27T07:00:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010f5c0 con 0x7f28f01041c0 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58794/0 (socket says 172.21.15.16:58794) 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28ef400640 1 -- 172.21.15.16:0/4254942349 learned_addr learned my addr 172.21.15.16:0/4254942349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28efe00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01041c0 0x7f28f0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.656+0000 7f28f5000640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 0x7f28f0102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 -- 172.21.15.16:0/4254942349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01041c0 msgr2=0x7f28f0103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01041c0 0x7f28f0103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 -- 172.21.15.16:0/4254942349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 msgr2=0x7f28f0102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 0x7f28f0102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 -- 172.21.15.16:0/4254942349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f0102d40 con 0x7f28f0106bb0 2024-05-27T07:00:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.657+0000 7f28ef400640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9f0 tx=0x7f28dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.659+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00cd80 con 0x7f28f0106bb0 2024-05-27T07:00:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.659+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28dc0054b0 con 0x7f28f0106bb0 2024-05-27T07:00:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.659+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc010430 con 0x7f28f0106bb0 2024-05-27T07:00:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.659+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a4190 con 0x7f28f0106bb0 2024-05-27T07:00:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.659+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a4600 con 0x7f28f0106bb0 2024-05-27T07:00:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.661+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f00ff660 con 0x7f28f0106bb0 2024-05-27T07:00:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.664+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 149464+0+0 (secure 0 0 0) 0x7f28dc0046c0 con 0x7f28f0106bb0 2024-05-27T07:00:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.666+0000 7f28ecc00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 0x7f28d40b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.667+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f28dc0e2670 con 0x7f28f0106bb0 2024-05-27T07:00:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.667+0000 7f28efe00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 0x7f28d40b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.667+0000 7f28efe00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 0x7f28d40b4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28e00057c0 tx=0x7f28e0031400 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.670+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28dc0aa190 con 0x7f28f0106bb0 2024-05-27T07:00:55.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.783+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 22) ==== 149823+0+0 (secure 0 0 0) 0x7f28dc0a5d20 con 0x7f28f0106bb0 2024-05-27T07:00:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.931+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28f00638c0 con 0x7f28f0106bb0 2024-05-27T07:00:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.932+0000 7f28ecc00640 1 -- 172.21.15.16:0/4254942349 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f28dc00f3c0 con 0x7f28f0106bb0 2024-05-27T07:00:55.924 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.934+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 msgr2=0x7f28d40b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.934+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 0x7f28d40b4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28e00057c0 tx=0x7f28e0031400 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.934+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 msgr2=0x7f28f01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.934+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9f0 tx=0x7f28dc00edf0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.934+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 shutdown_connections 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f28d40b2660 0x7f28d40b4b70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28f01096e0 0x7f28f0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0106bb0 0x7f28f01020d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 --2- 172.21.15.16:0/4254942349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01041c0 0x7f28f0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 >> 172.21.15.16:0/4254942349 conn(0x7f28f00fdb40 msgr2=0x7f28f0106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 shutdown_connections 2024-05-27T07:00:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:55.935+0000 7f28f5a00640 1 -- 172.21.15.16:0/4254942349 wait complete. 2024-05-27T07:00:55.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T07:00:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:55 smithi164 ceph-mon[35327]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:55 smithi164 ceph-mon[35327]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:55 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4259966872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:55 smithi164 ceph-mon[35327]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:55 smithi016 ceph-mon[31415]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:56.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:55 smithi016 ceph-mon[31415]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:55 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4259966872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:56.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:55 smithi016 ceph-mon[31415]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:55 smithi069 ceph-mon[35276]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:00:56.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:55 smithi069 ceph-mon[35276]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:55 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4259966872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:00:56.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:55 smithi069 ceph-mon[35276]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 -- 172.21.15.16:0/2913483299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 msgr2=0x7ff9701125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 --2- 172.21.15.16:0/2913483299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701125b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff968009be0 tx=0x7ff96802a6d0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 -- 172.21.15.16:0/2913483299 shutdown_connections 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 --2- 172.21.15.16:0/2913483299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701125b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 --2- 172.21.15.16:0/2913483299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 0x7ff97010fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 --2- 172.21.15.16:0/2913483299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9700754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 -- 172.21.15.16:0/2913483299 >> 172.21.15.16:0/2913483299 conn(0x7ff9700fec40 msgr2=0x7ff9701010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 -- 172.21.15.16:0/2913483299 shutdown_connections 2024-05-27T07:00:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.249+0000 7ff975600640 1 -- 172.21.15.16:0/2913483299 wait complete. 2024-05-27T07:00:56.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.250+0000 7ff975600640 1 Processor -- start 2024-05-27T07:00:56.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.250+0000 7ff975600640 1 -- start start 2024-05-27T07:00:56.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 0x7ff9701a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff970115a20 con 0x7ff970110190 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701158a0 con 0x7ff970075a00 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff975600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff970115ba0 con 0x7ff9700750b0 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff974c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44706/0 (socket says 172.21.15.16:44706) 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff96fe00640 1 -- 172.21.15.16:0/1634359987 learned_addr learned my addr 172.21.15.16:0/1634359987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.251+0000 7ff96f400640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 0x7ff9701a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 -- 172.21.15.16:0/1634359987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 msgr2=0x7ff9701a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 0x7ff9701a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 -- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 msgr2=0x7ff9701a7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701a7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 -- 172.21.15.16:0/1634359987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9701a81a0 con 0x7ff9700750b0 2024-05-27T07:00:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ccc0 con 0x7ff9700750b0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9600054b0 con 0x7ff9700750b0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701a8490 con 0x7ff9700750b0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960010430 con 0x7ff9700750b0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.252+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701a8990 con 0x7ff9700750b0 2024-05-27T07:00:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.254+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff970105e20 con 0x7ff9700750b0 2024-05-27T07:00:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.256+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 22) ==== 149823+0+0 (secure 0 0 0) 0x7ff9600046c0 con 0x7ff9700750b0 2024-05-27T07:00:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.258+0000 7ff96cc00640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 0x7ff9540b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.258+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff9600e2e70 con 0x7ff9700750b0 2024-05-27T07:00:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.258+0000 7ff96f400640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 0x7ff9540b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.259+0000 7ff96f400640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 0x7ff9540b4ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0096c0 tx=0x7ff95c009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.263+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9600aa350 con 0x7ff9700750b0 2024-05-27T07:00:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.530+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9700638c0 con 0x7ff9700750b0 2024-05-27T07:00:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.531+0000 7ff96cc00640 1 -- 172.21.15.16:0/1634359987 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff9600a9c50 con 0x7ff9700750b0 2024-05-27T07:00:56.523 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 msgr2=0x7ff9540b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 0x7ff9540b4ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0096c0 tx=0x7ff95c009260 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 msgr2=0x7ff9701a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 shutdown_connections 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff9540b29e0 0x7ff9540b4ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970110190 0x7ff9701a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970075a00 0x7ff9701a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 --2- 172.21.15.16:0/1634359987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9700750b0 0x7ff9701a0660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 >> 172.21.15.16:0/1634359987 conn(0x7ff9700fec40 msgr2=0x7ff970102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 shutdown_connections 2024-05-27T07:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.534+0000 7ff975600640 1 -- 172.21.15.16:0/1634359987 wait complete. 2024-05-27T07:00:56.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T07:00:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 -- 172.21.15.16:0/1917793683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 msgr2=0x7f2488105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 --2- 172.21.15.16:0/1917793683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f2488105270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2470009900 tx=0x7f247002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 -- 172.21.15.16:0/1917793683 shutdown_connections 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 --2- 172.21.15.16:0/1917793683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488109b60 0x7f248810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 --2- 172.21.15.16:0/1917793683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f2488105270 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 --2- 172.21.15.16:0/1917793683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488103c20 0x7f2488104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 -- 172.21.15.16:0/1917793683 >> 172.21.15.16:0/1917793683 conn(0x7f24880fee10 msgr2=0x7f24881012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 -- 172.21.15.16:0/1917793683 shutdown_connections 2024-05-27T07:00:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.842+0000 7f248ca00640 1 -- 172.21.15.16:0/1917793683 wait complete. 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.843+0000 7f248ca00640 1 Processor -- start 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.843+0000 7f248ca00640 1 -- start start 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f248819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488109b60 0x7f24881a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f248810f600 con 0x7f2488104e60 2024-05-27T07:00:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f248810f480 con 0x7f2488109b60 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f248ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f248810f780 con 0x7f2488103c20 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f247fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f248819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44720/0 (socket says 172.21.15.16:44720) 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 -- 172.21.15.16:0/3178272484 learned_addr learned my addr 172.21.15.16:0/3178272484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2486800640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488109b60 0x7f24881a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 -- 172.21.15.16:0/3178272484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488109b60 msgr2=0x7f24881a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488109b60 0x7f24881a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 -- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 msgr2=0x7f248819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.844+0000 7f2485e00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f248819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f2485e00640 1 -- 172.21.15.16:0/3178272484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24881a1d80 con 0x7f2488103c20 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f247fe00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f248819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f2485e00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f247800e980 tx=0x7f247800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247800cd80 con 0x7f2488103c20 2024-05-27T07:00:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24780108c0 con 0x7f2488103c20 2024-05-27T07:00:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24881a2070 con 0x7f2488103c20 2024-05-27T07:00:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478018840 con 0x7f2488103c20 2024-05-27T07:00:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.845+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24881a25c0 con 0x7f2488103c20 2024-05-27T07:00:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.846+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2488108970 con 0x7f2488103c20 2024-05-27T07:00:56.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.849+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 22) ==== 149823+0+0 (secure 0 0 0) 0x7f247801f020 con 0x7f2488103c20 2024-05-27T07:00:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.851+0000 7f247ea00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 0x7f246c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.851+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f24780e2df0 con 0x7f2488103c20 2024-05-27T07:00:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.851+0000 7f247fe00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 0x7f246c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.852+0000 7f247fe00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 0x7f246c0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f248819b2e0 tx=0x7f2470000bc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:56.853+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24780aa270 con 0x7f2488103c20 2024-05-27T07:00:56.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:56 smithi016 ceph-mon[31415]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-27T07:00:56.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:56 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/4254942349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:00:56.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:56 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1634359987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:00:57.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.083+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f248802ddb0 con 0x7f2488103c20 2024-05-27T07:00:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.088+0000 7f247ea00640 1 -- 172.21.15.16:0/3178272484 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f24780cae20 con 0x7f2488103c20 2024-05-27T07:00:57.081 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-27T07:00:57.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:56 smithi164 ceph-mon[35327]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-27T07:00:57.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:56 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/4254942349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:00:57.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:56 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1634359987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:00:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 msgr2=0x7f246c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 0x7f246c0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f248819b2e0 tx=0x7f2470000bc0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 msgr2=0x7f248819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f247800e980 tx=0x7f247800ee40 comp rx=0 tx=0).stop 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 shutdown_connections 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f246c0b29f0 0x7f246c0b4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2488109b60 0x7f24881a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e60 0x7f248819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 --2- 172.21.15.16:0/3178272484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2488103c20 0x7f248819a240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 >> 172.21.15.16:0/3178272484 conn(0x7f24880fee10 msgr2=0x7f2488105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 shutdown_connections 2024-05-27T07:00:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.092+0000 7f248ca00640 1 -- 172.21.15.16:0/3178272484 wait complete. 2024-05-27T07:00:57.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T07:00:57.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:56 smithi069 ceph-mon[35276]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-27T07:00:57.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:56 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/4254942349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:00:57.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:56 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1634359987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.382+0000 7fd5d5000640 1 -- 172.21.15.16:0/1172143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 msgr2=0x7fd5d01067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.382+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1172143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d01067c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009900 tx=0x7fd5c002a410 comp rx=0 tx=0).stop 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.382+0000 7fd5d5000640 1 -- 172.21.15.16:0/1172143188 shutdown_connections 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1172143188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d01098a0 0x7fd5d010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1172143188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d0106d90 0x7fd5d0109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1172143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d01067c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 -- 172.21.15.16:0/1172143188 >> 172.21.15.16:0/1172143188 conn(0x7fd5d00fdce0 msgr2=0x7fd5d01001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 -- 172.21.15.16:0/1172143188 shutdown_connections 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.383+0000 7fd5d5000640 1 -- 172.21.15.16:0/1172143188 wait complete. 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 Processor -- start 2024-05-27T07:00:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 -- start start 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d0106d90 0x7fd5d019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 0x7fd5d01a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010f610 con 0x7fd5d01043a0 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010f490 con 0x7fd5d01098a0 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.384+0000 7fd5d5000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010f790 con 0x7fd5d0106d90 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5ce800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 0x7fd5d01a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58866/0 (socket says 172.21.15.16:58866) 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 -- 172.21.15.16:0/1987817122 learned_addr learned my addr 172.21.15.16:0/1987817122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5c5400640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d0106d90 0x7fd5d019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 -- 172.21.15.16:0/1987817122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d0106d90 msgr2=0x7fd5d019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d0106d90 0x7fd5d019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 -- 172.21.15.16:0/1987817122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 msgr2=0x7fd5d01a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 0x7fd5d01a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5cde00640 1 -- 172.21.15.16:0/1987817122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01a3fc0 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.385+0000 7fd5ce800640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 0x7fd5d01a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5cde00640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005760 tx=0x7fd5c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c003f070 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c0031ba0 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0035030 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01a4250 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.386+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01a4790 con 0x7fd5d01043a0 2024-05-27T07:00:57.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.387+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d010b480 con 0x7fd5d01043a0 2024-05-27T07:00:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.390+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 149823+0+0 (secure 0 0 0) 0x7fd5c0042070 con 0x7fd5d01043a0 2024-05-27T07:00:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.391+0000 7fd5c7400640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 0x7fd5b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:00:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.391+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd5c01033a0 con 0x7fd5d01043a0 2024-05-27T07:00:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.391+0000 7fd5c5400640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 0x7fd5b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:00:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.392+0000 7fd5c5400640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 0x7fd5b40b4f80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5d019d520 tx=0x7fd5b8005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:00:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.396+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5c00ca1d0 con 0x7fd5d01043a0 2024-05-27T07:00:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.613+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd5d00638c0 con 0x7fd5d01043a0 2024-05-27T07:00:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.613+0000 7fd5c7400640 1 -- 172.21.15.16:0/1987817122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fd5c00ce4d0 con 0x7fd5d01043a0 2024-05-27T07:00:57.606 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.616+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 msgr2=0x7fd5b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.616+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 0x7fd5b40b4f80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5d019d520 tx=0x7fd5b8005f90 comp rx=0 tx=0).stop 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.616+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 msgr2=0x7fd5d019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.616+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005760 tx=0x7fd5c0003c10 comp rx=0 tx=0).stop 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 shutdown_connections 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5b40b2a70 0x7fd5b40b4f80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d01098a0 0x7fd5d01a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d0106d90 0x7fd5d019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 --2- 172.21.15.16:0/1987817122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d01043a0 0x7fd5d019c480 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:00:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 >> 172.21.15.16:0/1987817122 conn(0x7fd5d00fdce0 msgr2=0x7fd5d0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:00:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 shutdown_connections 2024-05-27T07:00:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:00:57.617+0000 7fd5d5000640 1 -- 172.21.15.16:0/1987817122 wait complete. 2024-05-27T07:00:57.635 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-27T07:00:57.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:00:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:00:58.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3178272484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1987817122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:58 smithi164 ceph-mon[35327]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:58.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3178272484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1987817122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:58 smithi016 ceph-mon[31415]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:58.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3178272484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1987817122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:00:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:58 smithi069 ceph-mon[35276]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:00:59.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:00:59 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:00:59 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-27T07:00:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:00:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:00:59.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:00:59 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.461+0000 7f16c1200700 1 -- 172.21.15.16:0/3584432078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc1109a0 msgr2=0x7f16bc112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.461+0000 7f16c1200700 1 --2- 172.21.15.16:0/3584432078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc1109a0 0x7f16bc112e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b0a0 tx=0x7f16b400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 -- 172.21.15.16:0/3584432078 shutdown_connections 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 --2- 172.21.15.16:0/3584432078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc1109a0 0x7f16bc112e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 --2- 172.21.15.16:0/3584432078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 0x7f16bc076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 --2- 172.21.15.16:0/3584432078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc076670 0x7f16bc110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 -- 172.21.15.16:0/3584432078 >> 172.21.15.16:0/3584432078 conn(0x7f16bc079d00 msgr2=0x7f16bc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 -- 172.21.15.16:0/3584432078 shutdown_connections 2024-05-27T07:01:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.462+0000 7f16c1200700 1 -- 172.21.15.16:0/3584432078 wait complete. 2024-05-27T07:01:00.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:01:00.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:01:00.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:01:00.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T07:01:00.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:00 smithi016 ceph-mon[31415]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:01:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.463+0000 7f16c1200700 1 Processor -- start 2024-05-27T07:01:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.463+0000 7f16c1200700 1 -- start start 2024-05-27T07:01:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 0x7f16bc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc1109a0 0x7f16bc1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc116380 con 0x7f16bc1109a0 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc116200 con 0x7f16bc076670 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16c1200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc116500 con 0x7f16bc075d20 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16baa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16baa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:54008/0 (socket says 172.21.15.16:54008) 2024-05-27T07:01:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16baa00700 1 -- 172.21.15.16:0/3446556567 learned_addr learned my addr 172.21.15.16:0/3446556567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16bb400700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 0x7f16bc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.464+0000 7f16bbe00700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc1109a0 0x7f16bc1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 -- 172.21.15.16:0/3446556567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 msgr2=0x7f16bc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 0x7f16bc19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 -- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc1109a0 msgr2=0x7f16bc1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc1109a0 0x7f16bc1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 -- 172.21.15.16:0/3446556567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1a1c40 con 0x7f16bc076670 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.465+0000 7f16baa00700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9a0 tx=0x7f16a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.466+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800cbd0 con 0x7f16bc076670 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.466+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16a800cd70 con 0x7f16bc076670 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.466+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc1a1f30 con 0x7f16bc076670 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.466+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a9850 con 0x7f16bc076670 2024-05-27T07:01:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.466+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8018680 con 0x7f16bc076670 2024-05-27T07:01:00.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.467+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c005330 con 0x7f16bc076670 2024-05-27T07:01:00.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.471+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f16a804ab90 con 0x7f16bc076670 2024-05-27T07:01:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.473+0000 7f16b3e00700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 0x7f16a00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.473+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f16a80e2e00 con 0x7f16bc076670 2024-05-27T07:01:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.473+0000 7f16bb400700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 0x7f16a00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.474+0000 7f16bb400700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 0x7f16a00b50a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009c60 tx=0x7f16ac009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.477+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16a80aa1a0 con 0x7f16bc076670 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T07:01:00.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:00 smithi164 ceph-mon[35327]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:01:00.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:01:00.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:01:00.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-27T07:01:00.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:00 smithi069 ceph-mon[35276]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:01:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.795+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169c005620 con 0x7f16bc076670 2024-05-27T07:01:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.796+0000 7f16b3e00700 1 -- 172.21.15.16:0/3446556567 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f16a80a9aa0 con 0x7f16bc076670 2024-05-27T07:01:00.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:00.789 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:01:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.799+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 msgr2=0x7f16a00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.799+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 0x7f16a00b50a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009c60 tx=0x7f16ac009260 comp rx=0 tx=0).stop 2024-05-27T07:01:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.799+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 msgr2=0x7f16bc19d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.799+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9a0 tx=0x7f16a800ecb0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 shutdown_connections 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f16a00b2b90 0x7f16a00b50a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16bc1109a0 0x7f16bc1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16bc075d20 0x7f16bc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 --2- 172.21.15.16:0/3446556567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16bc076670 0x7f16bc19d310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 >> 172.21.15.16:0/3446556567 conn(0x7f16bc079d00 msgr2=0x7f16bc100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 shutdown_connections 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:00.800+0000 7f16c1200700 1 -- 172.21.15.16:0/3446556567 wait complete. 2024-05-27T07:01:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:01:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:01 smithi164 ceph-mon[35327]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:01 smithi164 ceph-mon[35327]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:01 smithi164 ceph-mon[35327]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:01 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3446556567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:01.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:01 smithi016 ceph-mon[31415]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:01 smithi016 ceph-mon[31415]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:01 smithi016 ceph-mon[31415]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:01 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3446556567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:01.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:01 smithi069 ceph-mon[35276]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:01 smithi069 ceph-mon[35276]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:01 smithi069 ceph-mon[35276]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:01:01.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:01 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3446556567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:01.863 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-27T07:01:01.864 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-27T07:01:01.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 -- 172.21.15.16:0/3005065671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc10b5b0 msgr2=0x7f3fbc10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 --2- 172.21.15.16:0/3005065671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc10d9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009900 tx=0x7f3fa802a410 comp rx=0 tx=0).stop 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 -- 172.21.15.16:0/3005065671 shutdown_connections 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 --2- 172.21.15.16:0/3005065671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 0x7f3fbc110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 --2- 172.21.15.16:0/3005065671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc10d9d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 --2- 172.21.15.16:0/3005065671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc104b50 0x7f3fbc104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.153+0000 7f3fbb200640 1 -- 172.21.15.16:0/3005065671 >> 172.21.15.16:0/3005065671 conn(0x7f3fbc0780b0 msgr2=0x7f3fbc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.154+0000 7f3fbb200640 1 -- 172.21.15.16:0/3005065671 shutdown_connections 2024-05-27T07:01:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.154+0000 7f3fbb200640 1 -- 172.21.15.16:0/3005065671 wait complete. 2024-05-27T07:01:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.155+0000 7f3fbb200640 1 Processor -- start 2024-05-27T07:01:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.155+0000 7f3fbb200640 1 -- start start 2024-05-27T07:01:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 0x7f3fbc1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc113b00 con 0x7f3fbc104b50 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc113980 con 0x7f3fbc10b5b0 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fbb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc113c80 con 0x7f3fbc10dfa0 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fb9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fb9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37228/0 (socket says 172.21.15.16:37228) 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fb9e00640 1 -- 172.21.15.16:0/2189481525 learned_addr learned my addr 172.21.15.16:0/2189481525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fba800640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 0x7f3fbc1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.156+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 -- 172.21.15.16:0/2189481525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 msgr2=0x7f3fbc1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 0x7f3fbc1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 -- 172.21.15.16:0/2189481525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 msgr2=0x7f3fbc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 -- 172.21.15.16:0/2189481525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc1a1640 con 0x7f3fbc104b50 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.157+0000 7f3fb9e00640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00d950 tx=0x7f3fac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.161+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac017070 con 0x7f3fbc104b50 2024-05-27T07:01:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.161+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fac009c30 con 0x7f3fbc104b50 2024-05-27T07:01:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.161+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc1a1930 con 0x7f3fbc104b50 2024-05-27T07:01:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.161+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00f620 con 0x7f3fbc104b50 2024-05-27T07:01:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.161+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc1a92b0 con 0x7f3fbc104b50 2024-05-27T07:01:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.162+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fbc06a890 con 0x7f3fbc104b50 2024-05-27T07:01:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.167+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f3fac010460 con 0x7f3fbc104b50 2024-05-27T07:01:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.168+0000 7f3fb2a00640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 0x7f3fa00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.169+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f3fac0e2940 con 0x7f3fbc104b50 2024-05-27T07:01:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.169+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 0x7f3fa00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.170+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 0x7f3fa00b4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa8005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.171+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3fac0a9ce0 con 0x7f3fbc104b50 2024-05-27T07:01:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.397+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3fbc10f490 con 0x7f3fa00b2a70 2024-05-27T07:01:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.418+0000 7f3fb2a00640 1 -- 172.21.15.16:0/2189481525 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339519 (secure 0 0 0) 0x7f3fbc10f490 con 0x7f3fa00b2a70 2024-05-27T07:01:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T07:01:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.426+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 msgr2=0x7f3fa00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.426+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 0x7f3fa00b4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa8005a20 comp rx=0 tx=0).stop 2024-05-27T07:01:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.426+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 msgr2=0x7f3fbc19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.426+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00d950 tx=0x7f3fac00de10 comp rx=0 tx=0).stop 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 shutdown_connections 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa00b2a70 0x7f3fa00b4f80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fbc10dfa0 0x7f3fbc1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc10b5b0 0x7f3fbc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 --2- 172.21.15.16:0/2189481525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc104b50 0x7f3fbc19c7e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 >> 172.21.15.16:0/2189481525 conn(0x7f3fbc0780b0 msgr2=0x7f3fbc109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 shutdown_connections 2024-05-27T07:01:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.427+0000 7f3fbb200640 1 -- 172.21.15.16:0/2189481525 wait complete. 2024-05-27T07:01:02.447 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:01:01.677966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1203375,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2848,"num_read_kb":1729,"num_write":540,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":927,"ondisk_log_size":927,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386132,"kb_used_data":16160,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555692,"statfs":{"total":1151940427776,"available":1151545028608,"internally_reserved":0,"allocated":16547840,"data_stored":9950303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":210,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.968601"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804740+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T07:00:53.804740+0000","last_peered":"2024-05-27T07:00:53.804740+0000","last_clean":"2024-05-27T07:00:53.804740+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T07:00:53.804740+0000","last_undegraded":"2024-05-27T07:00:53.804740+0000","last_fullsized":"2024-05-27T07:00:53.804740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844593+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T07:00:53.844593+0000","last_peered":"2024-05-27T07:00:53.844593+0000","last_clean":"2024-05-27T07:00:53.844593+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T07:00:53.844593+0000","last_undegraded":"2024-05-27T07:00:53.844593+0000","last_fullsized":"2024-05-27T07:00:53.844593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844310+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T07:00:53.844310+0000","last_peered":"2024-05-27T07:00:53.844310+0000","last_clean":"2024-05-27T07:00:53.844310+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T07:00:53.844310+0000","last_undegraded":"2024-05-27T07:00:53.844310+0000","last_fullsized":"2024-05-27T07:00:53.844310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782918+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T07:00:53.782918+0000","last_peered":"2024-05-27T07:00:53.782918+0000","last_clean":"2024-05-27T07:00:53.782918+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T07:00:53.782918+0000","last_undegraded":"2024-05-27T07:00:53.782918+0000","last_fullsized":"2024-05-27T07:00:53.782918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790665+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T07:00:53.790665+0000","last_peered":"2024-05-27T07:00:53.790665+0000","last_clean":"2024-05-27T07:00:53.790665+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T07:00:53.790665+0000","last_undegraded":"2024-05-27T07:00:53.790665+0000","last_fullsized":"2024-05-27T07:00:53.790665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776643+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T07:00:53.776643+0000","last_peered":"2024-05-27T07:00:53.776643+0000","last_clean":"2024-05-27T07:00:53.776643+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T07:00:53.776643+0000","last_undegraded":"2024-05-27T07:00:53.776643+0000","last_fullsized":"2024-05-27T07:00:53.776643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200158+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T07:00:54.200158+0000","last_peered":"2024-05-27T07:00:54.200158+0000","last_clean":"2024-05-27T07:00:54.200158+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T07:00:54.200158+0000","last_undegraded":"2024-05-27T07:00:54.200158+0000","last_fullsized":"2024-05-27T07:00:54.200158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801179+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T07:00:53.801179+0000","last_peered":"2024-05-27T07:00:53.801179+0000","last_clean":"2024-05-27T07:00:53.801179+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T07:00:53.801179+0000","last_undegraded":"2024-05-27T07:00:53.801179+0000","last_fullsized":"2024-05-27T07:00:53.801179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651181+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T07:00:53.651181+0000","last_peered":"2024-05-27T07:00:53.651181+0000","last_clean":"2024-05-27T07:00:53.651181+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T07:00:53.651181+0000","last_undegraded":"2024-05-27T07:00:53.651181+0000","last_fullsized":"2024-05-27T07:00:53.651181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.777439+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T07:00:53.777439+0000","last_peered":"2024-05-27T07:00:53.777439+0000","last_clean":"2024-05-27T07:00:53.777439+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T07:00:53.777439+0000","last_undegraded":"2024-05-27T07:00:53.777439+0000","last_fullsized":"2024-05-27T07:00:53.777439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200612+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T07:00:54.200612+0000","last_peered":"2024-05-27T07:00:54.200612+0000","last_clean":"2024-05-27T07:00:54.200612+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T07:00:54.200612+0000","last_undegraded":"2024-05-27T07:00:54.200612+0000","last_fullsized":"2024-05-27T07:00:54.200612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783009+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T07:00:53.783009+0000","last_peered":"2024-05-27T07:00:53.783009+0000","last_clean":"2024-05-27T07:00:53.783009+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T07:00:53.783009+0000","last_undegraded":"2024-05-27T07:00:53.783009+0000","last_fullsized":"2024-05-27T07:00:53.783009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203914+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T07:00:54.203914+0000","last_peered":"2024-05-27T07:00:54.203914+0000","last_clean":"2024-05-27T07:00:54.203914+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T07:00:54.203914+0000","last_undegraded":"2024-05-27T07:00:54.203914+0000","last_fullsized":"2024-05-27T07:00:54.203914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843989+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T07:00:53.843989+0000","last_peered":"2024-05-27T07:00:53.843989+0000","last_clean":"2024-05-27T07:00:53.843989+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T07:00:53.843989+0000","last_undegraded":"2024-05-27T07:00:53.843989+0000","last_fullsized":"2024-05-27T07:00:53.843989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843868+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T07:00:53.843868+0000","last_peered":"2024-05-27T07:00:53.843868+0000","last_clean":"2024-05-27T07:00:53.843868+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T07:00:53.843868+0000","last_undegraded":"2024-05-27T07:00:53.843868+0000","last_fullsized":"2024-05-27T07:00:53.843868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649192+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T07:00:53.649192+0000","last_peered":"2024-05-27T07:00:53.649192+0000","last_clean":"2024-05-27T07:00:53.649192+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T07:00:53.649192+0000","last_undegraded":"2024-05-27T07:00:53.649192+0000","last_fullsized":"2024-05-27T07:00:53.649192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804415+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T07:00:53.804415+0000","last_peered":"2024-05-27T07:00:53.804415+0000","last_clean":"2024-05-27T07:00:53.804415+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T07:00:53.804415+0000","last_undegraded":"2024-05-27T07:00:53.804415+0000","last_fullsized":"2024-05-27T07:00:53.804415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648444+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T07:00:53.648444+0000","last_peered":"2024-05-27T07:00:53.648444+0000","last_clean":"2024-05-27T07:00:53.648444+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T07:00:53.648444+0000","last_undegraded":"2024-05-27T07:00:53.648444+0000","last_fullsized":"2024-05-27T07:00:53.648444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651138+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T07:00:53.651138+0000","last_peered":"2024-05-27T07:00:53.651138+0000","last_clean":"2024-05-27T07:00:53.651138+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T07:00:53.651138+0000","last_undegraded":"2024-05-27T07:00:53.651138+0000","last_fullsized":"2024-05-27T07:00:53.651138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649738+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T07:00:53.649738+0000","last_peered":"2024-05-27T07:00:53.649738+0000","last_clean":"2024-05-27T07:00:53.649738+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T07:00:53.649738+0000","last_undegraded":"2024-05-27T07:00:53.649738+0000","last_fullsized":"2024-05-27T07:00:53.649738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":319,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.213065+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T07:00:58.213065+0000","last_peered":"2024-05-27T07:00:58.213065+0000","last_clean":"2024-05-27T07:00:58.213065+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T07:00:58.213065+0000","last_undegraded":"2024-05-27T07:00:58.213065+0000","last_fullsized":"2024-05-27T07:00:58.213065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803028+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T07:00:53.803028+0000","last_peered":"2024-05-27T07:00:53.803028+0000","last_clean":"2024-05-27T07:00:53.803028+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T07:00:53.803028+0000","last_undegraded":"2024-05-27T07:00:53.803028+0000","last_fullsized":"2024-05-27T07:00:53.803028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.781717+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T07:00:53.781717+0000","last_peered":"2024-05-27T07:00:53.781717+0000","last_clean":"2024-05-27T07:00:53.781717+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T07:00:53.781717+0000","last_undegraded":"2024-05-27T07:00:53.781717+0000","last_fullsized":"2024-05-27T07:00:53.781717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803020+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T07:00:53.803020+0000","last_peered":"2024-05-27T07:00:53.803020+0000","last_clean":"2024-05-27T07:00:53.803020+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T07:00:53.803020+0000","last_undegraded":"2024-05-27T07:00:53.803020+0000","last_fullsized":"2024-05-27T07:00:53.803020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844590+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T07:00:53.844590+0000","last_peered":"2024-05-27T07:00:53.844590+0000","last_clean":"2024-05-27T07:00:53.844590+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T07:00:53.844590+0000","last_undegraded":"2024-05-27T07:00:53.844590+0000","last_fullsized":"2024-05-27T07:00:53.844590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782289+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T07:00:53.782289+0000","last_peered":"2024-05-27T07:00:53.782289+0000","last_clean":"2024-05-27T07:00:53.782289+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T07:00:53.782289+0000","last_undegraded":"2024-05-27T07:00:53.782289+0000","last_fullsized":"2024-05-27T07:00:53.782289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792226+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T07:00:53.792226+0000","last_peered":"2024-05-27T07:00:53.792226+0000","last_clean":"2024-05-27T07:00:53.792226+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T07:00:53.792226+0000","last_undegraded":"2024-05-27T07:00:53.792226+0000","last_fullsized":"2024-05-27T07:00:53.792226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650377+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T07:00:53.650377+0000","last_peered":"2024-05-27T07:00:53.650377+0000","last_clean":"2024-05-27T07:00:53.650377+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T07:00:53.650377+0000","last_undegraded":"2024-05-27T07:00:53.650377+0000","last_fullsized":"2024-05-27T07:00:53.650377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782034+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T07:00:53.782034+0000","last_peered":"2024-05-27T07:00:53.782034+0000","last_clean":"2024-05-27T07:00:53.782034+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T07:00:53.782034+0000","last_undegraded":"2024-05-27T07:00:53.782034+0000","last_fullsized":"2024-05-27T07:00:53.782034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791486+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T07:00:53.791486+0000","last_peered":"2024-05-27T07:00:53.791486+0000","last_clean":"2024-05-27T07:00:53.791486+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T07:00:53.791486+0000","last_undegraded":"2024-05-27T07:00:53.791486+0000","last_fullsized":"2024-05-27T07:00:53.791486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817046+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T07:00:53.817046+0000","last_peered":"2024-05-27T07:00:53.817046+0000","last_clean":"2024-05-27T07:00:53.817046+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T07:00:53.817046+0000","last_undegraded":"2024-05-27T07:00:53.817046+0000","last_fullsized":"2024-05-27T07:00:53.817046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650642+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T07:00:53.650642+0000","last_peered":"2024-05-27T07:00:53.650642+0000","last_clean":"2024-05-27T07:00:53.650642+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T07:00:53.650642+0000","last_undegraded":"2024-05-27T07:00:53.650642+0000","last_fullsized":"2024-05-27T07:00:53.650642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":315,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.248852+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T07:00:58.248852+0000","last_peered":"2024-05-27T07:00:58.248852+0000","last_clean":"2024-05-27T07:00:58.248852+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T07:00:58.248852+0000","last_undegraded":"2024-05-27T07:00:58.248852+0000","last_fullsized":"2024-05-27T07:00:58.248852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790375+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T07:00:53.790375+0000","last_peered":"2024-05-27T07:00:53.790375+0000","last_clean":"2024-05-27T07:00:53.790375+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T07:00:53.790375+0000","last_undegraded":"2024-05-27T07:00:53.790375+0000","last_fullsized":"2024-05-27T07:00:53.790375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650611+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T07:00:53.650611+0000","last_peered":"2024-05-27T07:00:53.650611+0000","last_clean":"2024-05-27T07:00:53.650611+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T07:00:53.650611+0000","last_undegraded":"2024-05-27T07:00:53.650611+0000","last_fullsized":"2024-05-27T07:00:53.650611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776438+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T07:00:53.776438+0000","last_peered":"2024-05-27T07:00:53.776438+0000","last_clean":"2024-05-27T07:00:53.776438+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T07:00:53.776438+0000","last_undegraded":"2024-05-27T07:00:53.776438+0000","last_fullsized":"2024-05-27T07:00:53.776438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647617+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T07:00:53.647617+0000","last_peered":"2024-05-27T07:00:53.647617+0000","last_clean":"2024-05-27T07:00:53.647617+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T07:00:53.647617+0000","last_undegraded":"2024-05-27T07:00:53.647617+0000","last_fullsized":"2024-05-27T07:00:53.647617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803766+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T07:00:53.803766+0000","last_peered":"2024-05-27T07:00:53.803766+0000","last_clean":"2024-05-27T07:00:53.803766+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T07:00:53.803766+0000","last_undegraded":"2024-05-27T07:00:53.803766+0000","last_fullsized":"2024-05-27T07:00:53.803766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203388+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T07:00:54.203388+0000","last_peered":"2024-05-27T07:00:54.203388+0000","last_clean":"2024-05-27T07:00:54.203388+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T07:00:54.203388+0000","last_undegraded":"2024-05-27T07:00:54.203388+0000","last_fullsized":"2024-05-27T07:00:54.203388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801019+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T07:00:53.801019+0000","last_peered":"2024-05-27T07:00:53.801019+0000","last_clean":"2024-05-27T07:00:53.801019+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T07:00:53.801019+0000","last_undegraded":"2024-05-27T07:00:53.801019+0000","last_fullsized":"2024-05-27T07:00:53.801019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844584+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T07:00:53.844584+0000","last_peered":"2024-05-27T07:00:53.844584+0000","last_clean":"2024-05-27T07:00:53.844584+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T07:00:53.844584+0000","last_undegraded":"2024-05-27T07:00:53.844584+0000","last_fullsized":"2024-05-27T07:00:53.844584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815380+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T07:00:53.815380+0000","last_peered":"2024-05-27T07:00:53.815380+0000","last_clean":"2024-05-27T07:00:53.815380+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T07:00:53.815380+0000","last_undegraded":"2024-05-27T07:00:53.815380+0000","last_fullsized":"2024-05-27T07:00:53.815380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.646214+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T07:00:53.646214+0000","last_peered":"2024-05-27T07:00:53.646214+0000","last_clean":"2024-05-27T07:00:53.646214+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T07:00:53.646214+0000","last_undegraded":"2024-05-27T07:00:53.646214+0000","last_fullsized":"2024-05-27T07:00:53.646214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801166+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T07:00:53.801166+0000","last_peered":"2024-05-27T07:00:53.801166+0000","last_clean":"2024-05-27T07:00:53.801166+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T07:00:53.801166+0000","last_undegraded":"2024-05-27T07:00:53.801166+0000","last_fullsized":"2024-05-27T07:00:53.801166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":565,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.250632+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T07:00:59.250632+0000","last_peered":"2024-05-27T07:00:59.250632+0000","last_clean":"2024-05-27T07:00:59.250632+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T07:00:59.250632+0000","last_undegraded":"2024-05-27T07:00:59.250632+0000","last_fullsized":"2024-05-27T07:00:59.250632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":510,"num_read_kb":510,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.652372+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T07:00:53.652372+0000","last_peered":"2024-05-27T07:00:53.652372+0000","last_clean":"2024-05-27T07:00:53.652372+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T07:00:53.652372+0000","last_undegraded":"2024-05-27T07:00:53.652372+0000","last_fullsized":"2024-05-27T07:00:53.652372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.645594+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T07:00:53.645594+0000","last_peered":"2024-05-27T07:00:53.645594+0000","last_clean":"2024-05-27T07:00:53.645594+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T07:00:53.645594+0000","last_undegraded":"2024-05-27T07:00:53.645594+0000","last_fullsized":"2024-05-27T07:00:53.645594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200781+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T07:00:54.200781+0000","last_peered":"2024-05-27T07:00:54.200781+0000","last_clean":"2024-05-27T07:00:54.200781+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T07:00:54.200781+0000","last_undegraded":"2024-05-27T07:00:54.200781+0000","last_fullsized":"2024-05-27T07:00:54.200781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844519+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T07:00:53.844519+0000","last_peered":"2024-05-27T07:00:53.844519+0000","last_clean":"2024-05-27T07:00:53.844519+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T07:00:53.844519+0000","last_undegraded":"2024-05-27T07:00:53.844519+0000","last_fullsized":"2024-05-27T07:00:53.844519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782729+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T07:00:53.782729+0000","last_peered":"2024-05-27T07:00:53.782729+0000","last_clean":"2024-05-27T07:00:53.782729+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T07:00:53.782729+0000","last_undegraded":"2024-05-27T07:00:53.782729+0000","last_fullsized":"2024-05-27T07:00:53.782729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816934+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T07:00:53.816934+0000","last_peered":"2024-05-27T07:00:53.816934+0000","last_clean":"2024-05-27T07:00:53.816934+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T07:00:53.816934+0000","last_undegraded":"2024-05-27T07:00:53.816934+0000","last_fullsized":"2024-05-27T07:00:53.816934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650831+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T07:00:53.650831+0000","last_peered":"2024-05-27T07:00:53.650831+0000","last_clean":"2024-05-27T07:00:53.650831+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T07:00:53.650831+0000","last_undegraded":"2024-05-27T07:00:53.650831+0000","last_fullsized":"2024-05-27T07:00:53.650831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647361+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T07:00:53.647361+0000","last_peered":"2024-05-27T07:00:53.647361+0000","last_clean":"2024-05-27T07:00:53.647361+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T07:00:53.647361+0000","last_undegraded":"2024-05-27T07:00:53.647361+0000","last_fullsized":"2024-05-27T07:00:53.647361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'46","reported_seq":636,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.269405+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T07:00:59.269405+0000","last_peered":"2024-05-27T07:00:59.269405+0000","last_clean":"2024-05-27T07:00:59.269405+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T07:00:59.269405+0000","last_undegraded":"2024-05-27T07:00:59.269405+0000","last_fullsized":"2024-05-27T07:00:59.269405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75230,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":526,"num_read_kb":336,"num_write":48,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791216+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T07:00:53.791216+0000","last_peered":"2024-05-27T07:00:53.791216+0000","last_clean":"2024-05-27T07:00:53.791216+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T07:00:53.791216+0000","last_undegraded":"2024-05-27T07:00:53.791216+0000","last_fullsized":"2024-05-27T07:00:53.791216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817811+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T07:00:53.817811+0000","last_peered":"2024-05-27T07:00:53.817811+0000","last_clean":"2024-05-27T07:00:53.817811+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T07:00:53.817811+0000","last_undegraded":"2024-05-27T07:00:53.817811+0000","last_fullsized":"2024-05-27T07:00:53.817811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650368+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T07:00:53.650368+0000","last_peered":"2024-05-27T07:00:53.650368+0000","last_clean":"2024-05-27T07:00:53.650368+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T07:00:53.650368+0000","last_undegraded":"2024-05-27T07:00:53.650368+0000","last_fullsized":"2024-05-27T07:00:53.650368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203525+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T07:00:54.203525+0000","last_peered":"2024-05-27T07:00:54.203525+0000","last_clean":"2024-05-27T07:00:54.203525+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T07:00:54.203525+0000","last_undegraded":"2024-05-27T07:00:54.203525+0000","last_fullsized":"2024-05-27T07:00:54.203525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803276+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T07:00:53.803276+0000","last_peered":"2024-05-27T07:00:53.803276+0000","last_clean":"2024-05-27T07:00:53.803276+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T07:00:53.803276+0000","last_undegraded":"2024-05-27T07:00:53.803276+0000","last_fullsized":"2024-05-27T07:00:53.803276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816169+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T07:00:53.816169+0000","last_peered":"2024-05-27T07:00:53.816169+0000","last_clean":"2024-05-27T07:00:53.816169+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T07:00:53.816169+0000","last_undegraded":"2024-05-27T07:00:53.816169+0000","last_fullsized":"2024-05-27T07:00:53.816169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200447+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T07:00:54.200447+0000","last_peered":"2024-05-27T07:00:54.200447+0000","last_clean":"2024-05-27T07:00:54.200447+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T07:00:54.200447+0000","last_undegraded":"2024-05-27T07:00:54.200447+0000","last_fullsized":"2024-05-27T07:00:54.200447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648912+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T07:00:53.648912+0000","last_peered":"2024-05-27T07:00:53.648912+0000","last_clean":"2024-05-27T07:00:53.648912+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T07:00:53.648912+0000","last_undegraded":"2024-05-27T07:00:53.648912+0000","last_fullsized":"2024-05-27T07:00:53.648912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266661+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T07:00:58.266661+0000","last_peered":"2024-05-27T07:00:58.266661+0000","last_clean":"2024-05-27T07:00:58.266661+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T07:00:58.266661+0000","last_undegraded":"2024-05-27T07:00:58.266661+0000","last_fullsized":"2024-05-27T07:00:58.266661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":8,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200387+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T07:00:54.200387+0000","last_peered":"2024-05-27T07:00:54.200387+0000","last_clean":"2024-05-27T07:00:54.200387+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T07:00:54.200387+0000","last_undegraded":"2024-05-27T07:00:54.200387+0000","last_fullsized":"2024-05-27T07:00:54.200387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204558+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T07:00:54.204558+0000","last_peered":"2024-05-27T07:00:54.204558+0000","last_clean":"2024-05-27T07:00:54.204558+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T07:00:54.204558+0000","last_undegraded":"2024-05-27T07:00:54.204558+0000","last_fullsized":"2024-05-27T07:00:54.204558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789874+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T07:00:53.789874+0000","last_peered":"2024-05-27T07:00:53.789874+0000","last_clean":"2024-05-27T07:00:53.789874+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T07:00:53.789874+0000","last_undegraded":"2024-05-27T07:00:53.789874+0000","last_fullsized":"2024-05-27T07:00:53.789874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815836+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T07:00:53.815836+0000","last_peered":"2024-05-27T07:00:53.815836+0000","last_clean":"2024-05-27T07:00:53.815836+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T07:00:53.815836+0000","last_undegraded":"2024-05-27T07:00:53.815836+0000","last_fullsized":"2024-05-27T07:00:53.815836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815656+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T07:00:53.815656+0000","last_peered":"2024-05-27T07:00:53.815656+0000","last_clean":"2024-05-27T07:00:53.815656+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T07:00:53.815656+0000","last_undegraded":"2024-05-27T07:00:53.815656+0000","last_fullsized":"2024-05-27T07:00:53.815656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792118+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T07:00:53.792118+0000","last_peered":"2024-05-27T07:00:53.792118+0000","last_clean":"2024-05-27T07:00:53.792118+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T07:00:53.792118+0000","last_undegraded":"2024-05-27T07:00:53.792118+0000","last_fullsized":"2024-05-27T07:00:53.792118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803956+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T07:00:53.803956+0000","last_peered":"2024-05-27T07:00:53.803956+0000","last_clean":"2024-05-27T07:00:53.803956+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T07:00:53.803956+0000","last_undegraded":"2024-05-27T07:00:53.803956+0000","last_fullsized":"2024-05-27T07:00:53.803956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790756+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T07:00:53.790756+0000","last_peered":"2024-05-27T07:00:53.790756+0000","last_clean":"2024-05-27T07:00:53.790756+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T07:00:53.790756+0000","last_undegraded":"2024-05-27T07:00:53.790756+0000","last_fullsized":"2024-05-27T07:00:53.790756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648046+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T07:00:53.648046+0000","last_peered":"2024-05-27T07:00:53.648046+0000","last_clean":"2024-05-27T07:00:53.648046+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T07:00:53.648046+0000","last_undegraded":"2024-05-27T07:00:53.648046+0000","last_fullsized":"2024-05-27T07:00:53.648046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776506+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T07:00:53.776506+0000","last_peered":"2024-05-27T07:00:53.776506+0000","last_clean":"2024-05-27T07:00:53.776506+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T07:00:53.776506+0000","last_undegraded":"2024-05-27T07:00:53.776506+0000","last_fullsized":"2024-05-27T07:00:53.776506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203958+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T07:00:54.203958+0000","last_peered":"2024-05-27T07:00:54.203958+0000","last_clean":"2024-05-27T07:00:54.203958+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T07:00:54.203958+0000","last_undegraded":"2024-05-27T07:00:54.203958+0000","last_fullsized":"2024-05-27T07:00:54.203958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'9","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.214416+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T07:00:58.214416+0000","last_peered":"2024-05-27T07:00:58.214416+0000","last_clean":"2024-05-27T07:00:58.214416+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T07:00:58.214416+0000","last_undegraded":"2024-05-27T07:00:58.214416+0000","last_fullsized":"2024-05-27T07:00:58.214416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776481+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T07:00:53.776481+0000","last_peered":"2024-05-27T07:00:53.776481+0000","last_clean":"2024-05-27T07:00:53.776481+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T07:00:53.776481+0000","last_undegraded":"2024-05-27T07:00:53.776481+0000","last_fullsized":"2024-05-27T07:00:53.776481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204687+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T07:00:54.204687+0000","last_peered":"2024-05-27T07:00:54.204687+0000","last_clean":"2024-05-27T07:00:54.204687+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T07:00:54.204687+0000","last_undegraded":"2024-05-27T07:00:54.204687+0000","last_fullsized":"2024-05-27T07:00:54.204687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815963+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T07:00:53.815963+0000","last_peered":"2024-05-27T07:00:53.815963+0000","last_clean":"2024-05-27T07:00:53.815963+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T07:00:53.815963+0000","last_undegraded":"2024-05-27T07:00:53.815963+0000","last_fullsized":"2024-05-27T07:00:53.815963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":311,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.264972+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T07:00:58.264972+0000","last_peered":"2024-05-27T07:00:58.264972+0000","last_clean":"2024-05-27T07:00:58.264972+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T07:00:58.264972+0000","last_undegraded":"2024-05-27T07:00:58.264972+0000","last_fullsized":"2024-05-27T07:00:58.264972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200533+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T07:00:54.200533+0000","last_peered":"2024-05-27T07:00:54.200533+0000","last_clean":"2024-05-27T07:00:54.200533+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T07:00:54.200533+0000","last_undegraded":"2024-05-27T07:00:54.200533+0000","last_fullsized":"2024-05-27T07:00:54.200533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801444+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T07:00:53.801444+0000","last_peered":"2024-05-27T07:00:53.801444+0000","last_clean":"2024-05-27T07:00:53.801444+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T07:00:53.801444+0000","last_undegraded":"2024-05-27T07:00:53.801444+0000","last_fullsized":"2024-05-27T07:00:53.801444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782485+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T07:00:53.782485+0000","last_peered":"2024-05-27T07:00:53.782485+0000","last_clean":"2024-05-27T07:00:53.782485+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T07:00:53.782485+0000","last_undegraded":"2024-05-27T07:00:53.782485+0000","last_fullsized":"2024-05-27T07:00:53.782485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:55.842641+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T07:00:55.842641+0000","last_peered":"2024-05-27T07:00:55.842641+0000","last_clean":"2024-05-27T07:00:55.842641+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T07:00:55.842641+0000","last_undegraded":"2024-05-27T07:00:55.842641+0000","last_fullsized":"2024-05-27T07:00:55.842641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816609+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T07:00:53.816609+0000","last_peered":"2024-05-27T07:00:53.816609+0000","last_clean":"2024-05-27T07:00:53.816609+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T07:00:53.816609+0000","last_undegraded":"2024-05-27T07:00:53.816609+0000","last_fullsized":"2024-05-27T07:00:53.816609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203145+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T07:00:54.203145+0000","last_peered":"2024-05-27T07:00:54.203145+0000","last_clean":"2024-05-27T07:00:54.203145+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T07:00:54.203145+0000","last_undegraded":"2024-05-27T07:00:54.203145+0000","last_fullsized":"2024-05-27T07:00:54.203145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200185+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T07:00:54.200185+0000","last_peered":"2024-05-27T07:00:54.200185+0000","last_clean":"2024-05-27T07:00:54.200185+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T07:00:54.200185+0000","last_undegraded":"2024-05-27T07:00:54.200185+0000","last_fullsized":"2024-05-27T07:00:54.200185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803223+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T07:00:53.803223+0000","last_peered":"2024-05-27T07:00:53.803223+0000","last_clean":"2024-05-27T07:00:53.803223+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T07:00:53.803223+0000","last_undegraded":"2024-05-27T07:00:53.803223+0000","last_fullsized":"2024-05-27T07:00:53.803223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791810+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T07:00:53.791810+0000","last_peered":"2024-05-27T07:00:53.791810+0000","last_clean":"2024-05-27T07:00:53.791810+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T07:00:53.791810+0000","last_undegraded":"2024-05-27T07:00:53.791810+0000","last_fullsized":"2024-05-27T07:00:53.791810+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204155+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T07:00:54.204155+0000","last_peered":"2024-05-27T07:00:54.204155+0000","last_clean":"2024-05-27T07:00:54.204155+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T07:00:54.204155+0000","last_undegraded":"2024-05-27T07:00:54.204155+0000","last_fullsized":"2024-05-27T07:00:54.204155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801014+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T07:00:53.801014+0000","last_peered":"2024-05-27T07:00:53.801014+0000","last_clean":"2024-05-27T07:00:53.801014+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T07:00:53.801014+0000","last_undegraded":"2024-05-27T07:00:53.801014+0000","last_fullsized":"2024-05-27T07:00:53.801014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'53","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.053173+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T07:00:54.053173+0000","last_peered":"2024-05-27T07:00:54.053173+0000","last_clean":"2024-05-27T07:00:54.053173+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T07:00:54.053173+0000","last_undegraded":"2024-05-27T07:00:54.053173+0000","last_fullsized":"2024-05-27T07:00:54.053173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":94,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802653+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T07:00:53.802653+0000","last_peered":"2024-05-27T07:00:53.802653+0000","last_clean":"2024-05-27T07:00:53.802653+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T07:00:53.802653+0000","last_undegraded":"2024-05-27T07:00:53.802653+0000","last_fullsized":"2024-05-27T07:00:53.802653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649305+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T07:00:53.649305+0000","last_peered":"2024-05-27T07:00:53.649305+0000","last_clean":"2024-05-27T07:00:53.649305+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T07:00:53.649305+0000","last_undegraded":"2024-05-27T07:00:53.649305+0000","last_fullsized":"2024-05-27T07:00:53.649305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783726+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T07:00:53.783726+0000","last_peered":"2024-05-27T07:00:53.783726+0000","last_clean":"2024-05-27T07:00:53.783726+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T07:00:53.783726+0000","last_undegraded":"2024-05-27T07:00:53.783726+0000","last_fullsized":"2024-05-27T07:00:53.783726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844741+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T07:00:53.844741+0000","last_peered":"2024-05-27T07:00:53.844741+0000","last_clean":"2024-05-27T07:00:53.844741+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T07:00:53.844741+0000","last_undegraded":"2024-05-27T07:00:53.844741+0000","last_fullsized":"2024-05-27T07:00:53.844741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783683+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T07:00:53.783683+0000","last_peered":"2024-05-27T07:00:53.783683+0000","last_clean":"2024-05-27T07:00:53.783683+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T07:00:53.783683+0000","last_undegraded":"2024-05-27T07:00:53.783683+0000","last_fullsized":"2024-05-27T07:00:53.783683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649444+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T07:00:53.649444+0000","last_peered":"2024-05-27T07:00:53.649444+0000","last_clean":"2024-05-27T07:00:53.649444+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T07:00:53.649444+0000","last_undegraded":"2024-05-27T07:00:53.649444+0000","last_fullsized":"2024-05-27T07:00:53.649444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789897+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T07:00:53.789897+0000","last_peered":"2024-05-27T07:00:53.789897+0000","last_clean":"2024-05-27T07:00:53.789897+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T07:00:53.789897+0000","last_undegraded":"2024-05-27T07:00:53.789897+0000","last_fullsized":"2024-05-27T07:00:53.789897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649098+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T07:00:53.649098+0000","last_peered":"2024-05-27T07:00:53.649098+0000","last_clean":"2024-05-27T07:00:53.649098+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T07:00:53.649098+0000","last_undegraded":"2024-05-27T07:00:53.649098+0000","last_fullsized":"2024-05-27T07:00:53.649098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801357+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T07:00:53.801357+0000","last_peered":"2024-05-27T07:00:53.801357+0000","last_clean":"2024-05-27T07:00:53.801357+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T07:00:53.801357+0000","last_undegraded":"2024-05-27T07:00:53.801357+0000","last_fullsized":"2024-05-27T07:00:53.801357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803270+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T07:00:53.803270+0000","last_peered":"2024-05-27T07:00:53.803270+0000","last_clean":"2024-05-27T07:00:53.803270+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T07:00:53.803270+0000","last_undegraded":"2024-05-27T07:00:53.803270+0000","last_fullsized":"2024-05-27T07:00:53.803270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783228+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T07:00:53.783228+0000","last_peered":"2024-05-27T07:00:53.783228+0000","last_clean":"2024-05-27T07:00:53.783228+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T07:00:53.783228+0000","last_undegraded":"2024-05-27T07:00:53.783228+0000","last_fullsized":"2024-05-27T07:00:53.783228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649930+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T07:00:53.649930+0000","last_peered":"2024-05-27T07:00:53.649930+0000","last_clean":"2024-05-27T07:00:53.649930+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T07:00:53.649930+0000","last_undegraded":"2024-05-27T07:00:53.649930+0000","last_fullsized":"2024-05-27T07:00:53.649930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801666+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T07:00:53.801666+0000","last_peered":"2024-05-27T07:00:53.801666+0000","last_clean":"2024-05-27T07:00:53.801666+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T07:00:53.801666+0000","last_undegraded":"2024-05-27T07:00:53.801666+0000","last_fullsized":"2024-05-27T07:00:53.801666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783466+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T07:00:53.783466+0000","last_peered":"2024-05-27T07:00:53.783466+0000","last_clean":"2024-05-27T07:00:53.783466+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T07:00:53.783466+0000","last_undegraded":"2024-05-27T07:00:53.783466+0000","last_fullsized":"2024-05-27T07:00:53.783466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790516+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T07:00:53.790516+0000","last_peered":"2024-05-27T07:00:53.790516+0000","last_clean":"2024-05-27T07:00:53.790516+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T07:00:53.790516+0000","last_undegraded":"2024-05-27T07:00:53.790516+0000","last_fullsized":"2024-05-27T07:00:53.790516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200234+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T07:00:54.200234+0000","last_peered":"2024-05-27T07:00:54.200234+0000","last_clean":"2024-05-27T07:00:54.200234+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T07:00:54.200234+0000","last_undegraded":"2024-05-27T07:00:54.200234+0000","last_fullsized":"2024-05-27T07:00:54.200234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649466+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T07:00:53.649466+0000","last_peered":"2024-05-27T07:00:53.649466+0000","last_clean":"2024-05-27T07:00:53.649466+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T07:00:53.649466+0000","last_undegraded":"2024-05-27T07:00:53.649466+0000","last_fullsized":"2024-05-27T07:00:53.649466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266640+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T07:00:58.266640+0000","last_peered":"2024-05-27T07:00:58.266640+0000","last_clean":"2024-05-27T07:00:58.266640+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T07:00:58.266640+0000","last_undegraded":"2024-05-27T07:00:58.266640+0000","last_fullsized":"2024-05-27T07:00:58.266640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":7,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200900+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T07:00:54.200900+0000","last_peered":"2024-05-27T07:00:54.200900+0000","last_clean":"2024-05-27T07:00:54.200900+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T07:00:54.200900+0000","last_undegraded":"2024-05-27T07:00:54.200900+0000","last_fullsized":"2024-05-27T07:00:54.200900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844028+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T07:00:53.844028+0000","last_peered":"2024-05-27T07:00:53.844028+0000","last_clean":"2024-05-27T07:00:53.844028+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T07:00:53.844028+0000","last_undegraded":"2024-05-27T07:00:53.844028+0000","last_fullsized":"2024-05-27T07:00:53.844028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.042149+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T07:00:54.042149+0000","last_peered":"2024-05-27T07:00:54.042149+0000","last_clean":"2024-05-27T07:00:54.042149+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T07:00:54.042149+0000","last_undegraded":"2024-05-27T07:00:54.042149+0000","last_fullsized":"2024-05-27T07:00:54.042149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200622+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T07:00:54.200622+0000","last_peered":"2024-05-27T07:00:54.200622+0000","last_clean":"2024-05-27T07:00:54.200622+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T07:00:54.200622+0000","last_undegraded":"2024-05-27T07:00:54.200622+0000","last_fullsized":"2024-05-27T07:00:54.200622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815204+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T07:00:53.815204+0000","last_peered":"2024-05-27T07:00:53.815204+0000","last_clean":"2024-05-27T07:00:53.815204+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T07:00:53.815204+0000","last_undegraded":"2024-05-27T07:00:53.815204+0000","last_fullsized":"2024-05-27T07:00:53.815204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204413+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T07:00:54.204413+0000","last_peered":"2024-05-27T07:00:54.204413+0000","last_clean":"2024-05-27T07:00:54.204413+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T07:00:54.204413+0000","last_undegraded":"2024-05-27T07:00:54.204413+0000","last_fullsized":"2024-05-27T07:00:54.204413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200845+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T07:00:54.200845+0000","last_peered":"2024-05-27T07:00:54.200845+0000","last_clean":"2024-05-27T07:00:54.200845+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T07:00:54.200845+0000","last_undegraded":"2024-05-27T07:00:54.200845+0000","last_fullsized":"2024-05-27T07:00:54.200845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803485+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T07:00:53.803485+0000","last_peered":"2024-05-27T07:00:53.803485+0000","last_clean":"2024-05-27T07:00:53.803485+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T07:00:53.803485+0000","last_undegraded":"2024-05-27T07:00:53.803485+0000","last_fullsized":"2024-05-27T07:00:53.803485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650051+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T07:00:53.650051+0000","last_peered":"2024-05-27T07:00:53.650051+0000","last_clean":"2024-05-27T07:00:53.650051+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T07:00:53.650051+0000","last_undegraded":"2024-05-27T07:00:53.650051+0000","last_fullsized":"2024-05-27T07:00:53.650051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649601+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T07:00:53.649601+0000","last_peered":"2024-05-27T07:00:53.649601+0000","last_clean":"2024-05-27T07:00:53.649601+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T07:00:53.649601+0000","last_undegraded":"2024-05-27T07:00:53.649601+0000","last_fullsized":"2024-05-27T07:00:53.649601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844301+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T07:00:53.844301+0000","last_peered":"2024-05-27T07:00:53.844301+0000","last_clean":"2024-05-27T07:00:53.844301+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T07:00:53.844301+0000","last_undegraded":"2024-05-27T07:00:53.844301+0000","last_fullsized":"2024-05-27T07:00:53.844301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204340+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T07:00:54.204340+0000","last_peered":"2024-05-27T07:00:54.204340+0000","last_clean":"2024-05-27T07:00:54.204340+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T07:00:54.204340+0000","last_undegraded":"2024-05-27T07:00:54.204340+0000","last_fullsized":"2024-05-27T07:00:54.204340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804068+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T07:00:53.804068+0000","last_peered":"2024-05-27T07:00:53.804068+0000","last_clean":"2024-05-27T07:00:53.804068+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T07:00:53.804068+0000","last_undegraded":"2024-05-27T07:00:53.804068+0000","last_fullsized":"2024-05-27T07:00:53.804068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651709+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T07:00:53.651709+0000","last_peered":"2024-05-27T07:00:53.651709+0000","last_clean":"2024-05-27T07:00:53.651709+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T07:00:53.651709+0000","last_undegraded":"2024-05-27T07:00:53.651709+0000","last_fullsized":"2024-05-27T07:00:53.651709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802724+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T07:00:53.802724+0000","last_peered":"2024-05-27T07:00:53.802724+0000","last_clean":"2024-05-27T07:00:53.802724+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T07:00:53.802724+0000","last_undegraded":"2024-05-27T07:00:53.802724+0000","last_fullsized":"2024-05-27T07:00:53.802724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203093+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T07:00:54.203093+0000","last_peered":"2024-05-27T07:00:54.203093+0000","last_clean":"2024-05-27T07:00:54.203093+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T07:00:54.203093+0000","last_undegraded":"2024-05-27T07:00:54.203093+0000","last_fullsized":"2024-05-27T07:00:54.203093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801509+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T07:00:53.801509+0000","last_peered":"2024-05-27T07:00:53.801509+0000","last_clean":"2024-05-27T07:00:53.801509+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T07:00:53.801509+0000","last_undegraded":"2024-05-27T07:00:53.801509+0000","last_fullsized":"2024-05-27T07:00:53.801509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650494+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T07:00:53.650494+0000","last_peered":"2024-05-27T07:00:53.650494+0000","last_clean":"2024-05-27T07:00:53.650494+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T07:00:53.650494+0000","last_undegraded":"2024-05-27T07:00:53.650494+0000","last_fullsized":"2024-05-27T07:00:53.650494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203784+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T07:00:54.203784+0000","last_peered":"2024-05-27T07:00:54.203784+0000","last_clean":"2024-05-27T07:00:54.203784+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T07:00:54.203784+0000","last_undegraded":"2024-05-27T07:00:54.203784+0000","last_fullsized":"2024-05-27T07:00:54.203784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803772+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T07:00:53.803772+0000","last_peered":"2024-05-27T07:00:53.803772+0000","last_clean":"2024-05-27T07:00:53.803772+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T07:00:53.803772+0000","last_undegraded":"2024-05-27T07:00:53.803772+0000","last_fullsized":"2024-05-27T07:00:53.803772+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612773,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2704,"num_read_kb":1481,"num_write":403,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1838319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678106,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1454080,"data_stored":925987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874337,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28876,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716276,"statfs":{"total":95995035648,"available":95965466624,"internally_reserved":0,"allocated":1904640,"data_stored":1326417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":928206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234095,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":531272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430328,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1679360,"data_stored":1131876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626559,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":527814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":604074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":529661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":530730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1050922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986318,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":607509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822791,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1255835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":396215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:01:02.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 KiB/s wr, 19 op/s 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.583 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 KiB/s wr, 19 op/s 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: Updating smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 KiB/s wr, 19 op/s 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 -- 172.21.15.16:0/3526209392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 msgr2=0x7faf5c10fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 --2- 172.21.15.16:0/3526209392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c10fd80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faf48009900 tx=0x7faf4802a480 comp rx=0 tx=0).stop 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 -- 172.21.15.16:0/3526209392 shutdown_connections 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 --2- 172.21.15.16:0/3526209392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 0x7faf5c1126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 --2- 172.21.15.16:0/3526209392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c10fd80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 --2- 172.21.15.16:0/3526209392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 0x7faf5c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 -- 172.21.15.16:0/3526209392 >> 172.21.15.16:0/3526209392 conn(0x7faf5c079140 msgr2=0x7faf5c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 -- 172.21.15.16:0/3526209392 shutdown_connections 2024-05-27T07:01:02.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.757+0000 7faf5be00640 1 -- 172.21.15.16:0/3526209392 wait complete. 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.758+0000 7faf5be00640 1 Processor -- start 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 -- start start 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 0x7faf5c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 0x7faf5c1a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c115ce0 con 0x7faf5c10d960 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c115b60 con 0x7faf5c075400 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c115e60 con 0x7faf5c1102c0 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37248/0 (socket says 172.21.15.16:37248) 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.759+0000 7faf5a000640 1 -- 172.21.15.16:0/1555786724 learned_addr learned my addr 172.21.15.16:0/1555786724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5b400640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 0x7faf5c1a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5aa00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 0x7faf5c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5a000640 1 -- 172.21.15.16:0/1555786724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 msgr2=0x7faf5c1a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5a000640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 0x7faf5c1a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5a000640 1 -- 172.21.15.16:0/1555786724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 msgr2=0x7faf5c19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5a000640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 0x7faf5c19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.760+0000 7faf5a000640 1 -- 172.21.15.16:0/1555786724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5c1a15a0 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf5a000640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf48005940 tx=0x7faf48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4803f030 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf48004440 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5c1a1830 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48035070 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.761+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5c1a1d20 con 0x7faf5c10d960 2024-05-27T07:01:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.763+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5c069d00 con 0x7faf5c10d960 2024-05-27T07:01:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.767+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7faf48042030 con 0x7faf5c10d960 2024-05-27T07:01:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.769+0000 7faf53400640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 0x7faf400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.769+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7faf48102fc0 con 0x7faf5c10d960 2024-05-27T07:01:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.769+0000 7faf5aa00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 0x7faf400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.770+0000 7faf5aa00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 0x7faf400b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00bbe0 tx=0x7faf4c00b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.774+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf480c9de0 con 0x7faf5c10d960 2024-05-27T07:01:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:02.994+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf5c102850 con 0x7faf400b2a70 2024-05-27T07:01:03.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.016+0000 7faf53400640 1 -- 172.21.15.16:0/1555786724 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339519 (secure 0 0 0) 0x7faf5c102850 con 0x7faf400b2a70 2024-05-27T07:01:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T07:01:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 msgr2=0x7faf400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 0x7faf400b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00bbe0 tx=0x7faf4c00b390 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 msgr2=0x7faf5c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf48005940 tx=0x7faf48004040 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 shutdown_connections 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7faf400b2a70 0x7faf400b4f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c1102c0 0x7faf5c1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5c10d960 0x7faf5c19cca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 --2- 172.21.15.16:0/1555786724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf5c075400 0x7faf5c19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.023+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 >> 172.21.15.16:0/1555786724 conn(0x7faf5c079140 msgr2=0x7faf5c102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.024+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 shutdown_connections 2024-05-27T07:01:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.024+0000 7faf5be00640 1 -- 172.21.15.16:0/1555786724 wait complete. 2024-05-27T07:01:03.044 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:01:01.677966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1203375,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2848,"num_read_kb":1729,"num_write":540,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":927,"ondisk_log_size":927,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386132,"kb_used_data":16160,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555692,"statfs":{"total":1151940427776,"available":1151545028608,"internally_reserved":0,"allocated":16547840,"data_stored":9950303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":210,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.968601"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804740+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T07:00:53.804740+0000","last_peered":"2024-05-27T07:00:53.804740+0000","last_clean":"2024-05-27T07:00:53.804740+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T07:00:53.804740+0000","last_undegraded":"2024-05-27T07:00:53.804740+0000","last_fullsized":"2024-05-27T07:00:53.804740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844593+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T07:00:53.844593+0000","last_peered":"2024-05-27T07:00:53.844593+0000","last_clean":"2024-05-27T07:00:53.844593+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T07:00:53.844593+0000","last_undegraded":"2024-05-27T07:00:53.844593+0000","last_fullsized":"2024-05-27T07:00:53.844593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844310+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T07:00:53.844310+0000","last_peered":"2024-05-27T07:00:53.844310+0000","last_clean":"2024-05-27T07:00:53.844310+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T07:00:53.844310+0000","last_undegraded":"2024-05-27T07:00:53.844310+0000","last_fullsized":"2024-05-27T07:00:53.844310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782918+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T07:00:53.782918+0000","last_peered":"2024-05-27T07:00:53.782918+0000","last_clean":"2024-05-27T07:00:53.782918+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T07:00:53.782918+0000","last_undegraded":"2024-05-27T07:00:53.782918+0000","last_fullsized":"2024-05-27T07:00:53.782918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790665+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T07:00:53.790665+0000","last_peered":"2024-05-27T07:00:53.790665+0000","last_clean":"2024-05-27T07:00:53.790665+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T07:00:53.790665+0000","last_undegraded":"2024-05-27T07:00:53.790665+0000","last_fullsized":"2024-05-27T07:00:53.790665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776643+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T07:00:53.776643+0000","last_peered":"2024-05-27T07:00:53.776643+0000","last_clean":"2024-05-27T07:00:53.776643+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T07:00:53.776643+0000","last_undegraded":"2024-05-27T07:00:53.776643+0000","last_fullsized":"2024-05-27T07:00:53.776643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200158+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T07:00:54.200158+0000","last_peered":"2024-05-27T07:00:54.200158+0000","last_clean":"2024-05-27T07:00:54.200158+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T07:00:54.200158+0000","last_undegraded":"2024-05-27T07:00:54.200158+0000","last_fullsized":"2024-05-27T07:00:54.200158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801179+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T07:00:53.801179+0000","last_peered":"2024-05-27T07:00:53.801179+0000","last_clean":"2024-05-27T07:00:53.801179+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T07:00:53.801179+0000","last_undegraded":"2024-05-27T07:00:53.801179+0000","last_fullsized":"2024-05-27T07:00:53.801179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651181+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T07:00:53.651181+0000","last_peered":"2024-05-27T07:00:53.651181+0000","last_clean":"2024-05-27T07:00:53.651181+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T07:00:53.651181+0000","last_undegraded":"2024-05-27T07:00:53.651181+0000","last_fullsized":"2024-05-27T07:00:53.651181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.777439+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T07:00:53.777439+0000","last_peered":"2024-05-27T07:00:53.777439+0000","last_clean":"2024-05-27T07:00:53.777439+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T07:00:53.777439+0000","last_undegraded":"2024-05-27T07:00:53.777439+0000","last_fullsized":"2024-05-27T07:00:53.777439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200612+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T07:00:54.200612+0000","last_peered":"2024-05-27T07:00:54.200612+0000","last_clean":"2024-05-27T07:00:54.200612+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T07:00:54.200612+0000","last_undegraded":"2024-05-27T07:00:54.200612+0000","last_fullsized":"2024-05-27T07:00:54.200612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783009+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T07:00:53.783009+0000","last_peered":"2024-05-27T07:00:53.783009+0000","last_clean":"2024-05-27T07:00:53.783009+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T07:00:53.783009+0000","last_undegraded":"2024-05-27T07:00:53.783009+0000","last_fullsized":"2024-05-27T07:00:53.783009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203914+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T07:00:54.203914+0000","last_peered":"2024-05-27T07:00:54.203914+0000","last_clean":"2024-05-27T07:00:54.203914+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T07:00:54.203914+0000","last_undegraded":"2024-05-27T07:00:54.203914+0000","last_fullsized":"2024-05-27T07:00:54.203914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843989+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T07:00:53.843989+0000","last_peered":"2024-05-27T07:00:53.843989+0000","last_clean":"2024-05-27T07:00:53.843989+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T07:00:53.843989+0000","last_undegraded":"2024-05-27T07:00:53.843989+0000","last_fullsized":"2024-05-27T07:00:53.843989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843868+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T07:00:53.843868+0000","last_peered":"2024-05-27T07:00:53.843868+0000","last_clean":"2024-05-27T07:00:53.843868+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T07:00:53.843868+0000","last_undegraded":"2024-05-27T07:00:53.843868+0000","last_fullsized":"2024-05-27T07:00:53.843868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649192+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T07:00:53.649192+0000","last_peered":"2024-05-27T07:00:53.649192+0000","last_clean":"2024-05-27T07:00:53.649192+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T07:00:53.649192+0000","last_undegraded":"2024-05-27T07:00:53.649192+0000","last_fullsized":"2024-05-27T07:00:53.649192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804415+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T07:00:53.804415+0000","last_peered":"2024-05-27T07:00:53.804415+0000","last_clean":"2024-05-27T07:00:53.804415+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T07:00:53.804415+0000","last_undegraded":"2024-05-27T07:00:53.804415+0000","last_fullsized":"2024-05-27T07:00:53.804415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648444+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T07:00:53.648444+0000","last_peered":"2024-05-27T07:00:53.648444+0000","last_clean":"2024-05-27T07:00:53.648444+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T07:00:53.648444+0000","last_undegraded":"2024-05-27T07:00:53.648444+0000","last_fullsized":"2024-05-27T07:00:53.648444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651138+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T07:00:53.651138+0000","last_peered":"2024-05-27T07:00:53.651138+0000","last_clean":"2024-05-27T07:00:53.651138+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T07:00:53.651138+0000","last_undegraded":"2024-05-27T07:00:53.651138+0000","last_fullsized":"2024-05-27T07:00:53.651138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649738+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T07:00:53.649738+0000","last_peered":"2024-05-27T07:00:53.649738+0000","last_clean":"2024-05-27T07:00:53.649738+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T07:00:53.649738+0000","last_undegraded":"2024-05-27T07:00:53.649738+0000","last_fullsized":"2024-05-27T07:00:53.649738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":319,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.213065+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T07:00:58.213065+0000","last_peered":"2024-05-27T07:00:58.213065+0000","last_clean":"2024-05-27T07:00:58.213065+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T07:00:58.213065+0000","last_undegraded":"2024-05-27T07:00:58.213065+0000","last_fullsized":"2024-05-27T07:00:58.213065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803028+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T07:00:53.803028+0000","last_peered":"2024-05-27T07:00:53.803028+0000","last_clean":"2024-05-27T07:00:53.803028+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T07:00:53.803028+0000","last_undegraded":"2024-05-27T07:00:53.803028+0000","last_fullsized":"2024-05-27T07:00:53.803028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.781717+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T07:00:53.781717+0000","last_peered":"2024-05-27T07:00:53.781717+0000","last_clean":"2024-05-27T07:00:53.781717+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T07:00:53.781717+0000","last_undegraded":"2024-05-27T07:00:53.781717+0000","last_fullsized":"2024-05-27T07:00:53.781717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803020+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T07:00:53.803020+0000","last_peered":"2024-05-27T07:00:53.803020+0000","last_clean":"2024-05-27T07:00:53.803020+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T07:00:53.803020+0000","last_undegraded":"2024-05-27T07:00:53.803020+0000","last_fullsized":"2024-05-27T07:00:53.803020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844590+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T07:00:53.844590+0000","last_peered":"2024-05-27T07:00:53.844590+0000","last_clean":"2024-05-27T07:00:53.844590+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T07:00:53.844590+0000","last_undegraded":"2024-05-27T07:00:53.844590+0000","last_fullsized":"2024-05-27T07:00:53.844590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782289+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T07:00:53.782289+0000","last_peered":"2024-05-27T07:00:53.782289+0000","last_clean":"2024-05-27T07:00:53.782289+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T07:00:53.782289+0000","last_undegraded":"2024-05-27T07:00:53.782289+0000","last_fullsized":"2024-05-27T07:00:53.782289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792226+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T07:00:53.792226+0000","last_peered":"2024-05-27T07:00:53.792226+0000","last_clean":"2024-05-27T07:00:53.792226+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T07:00:53.792226+0000","last_undegraded":"2024-05-27T07:00:53.792226+0000","last_fullsized":"2024-05-27T07:00:53.792226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650377+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T07:00:53.650377+0000","last_peered":"2024-05-27T07:00:53.650377+0000","last_clean":"2024-05-27T07:00:53.650377+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T07:00:53.650377+0000","last_undegraded":"2024-05-27T07:00:53.650377+0000","last_fullsized":"2024-05-27T07:00:53.650377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782034+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T07:00:53.782034+0000","last_peered":"2024-05-27T07:00:53.782034+0000","last_clean":"2024-05-27T07:00:53.782034+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T07:00:53.782034+0000","last_undegraded":"2024-05-27T07:00:53.782034+0000","last_fullsized":"2024-05-27T07:00:53.782034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791486+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T07:00:53.791486+0000","last_peered":"2024-05-27T07:00:53.791486+0000","last_clean":"2024-05-27T07:00:53.791486+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T07:00:53.791486+0000","last_undegraded":"2024-05-27T07:00:53.791486+0000","last_fullsized":"2024-05-27T07:00:53.791486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817046+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T07:00:53.817046+0000","last_peered":"2024-05-27T07:00:53.817046+0000","last_clean":"2024-05-27T07:00:53.817046+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T07:00:53.817046+0000","last_undegraded":"2024-05-27T07:00:53.817046+0000","last_fullsized":"2024-05-27T07:00:53.817046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650642+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T07:00:53.650642+0000","last_peered":"2024-05-27T07:00:53.650642+0000","last_clean":"2024-05-27T07:00:53.650642+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T07:00:53.650642+0000","last_undegraded":"2024-05-27T07:00:53.650642+0000","last_fullsized":"2024-05-27T07:00:53.650642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":315,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.248852+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T07:00:58.248852+0000","last_peered":"2024-05-27T07:00:58.248852+0000","last_clean":"2024-05-27T07:00:58.248852+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T07:00:58.248852+0000","last_undegraded":"2024-05-27T07:00:58.248852+0000","last_fullsized":"2024-05-27T07:00:58.248852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790375+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T07:00:53.790375+0000","last_peered":"2024-05-27T07:00:53.790375+0000","last_clean":"2024-05-27T07:00:53.790375+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T07:00:53.790375+0000","last_undegraded":"2024-05-27T07:00:53.790375+0000","last_fullsized":"2024-05-27T07:00:53.790375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650611+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T07:00:53.650611+0000","last_peered":"2024-05-27T07:00:53.650611+0000","last_clean":"2024-05-27T07:00:53.650611+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T07:00:53.650611+0000","last_undegraded":"2024-05-27T07:00:53.650611+0000","last_fullsized":"2024-05-27T07:00:53.650611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776438+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T07:00:53.776438+0000","last_peered":"2024-05-27T07:00:53.776438+0000","last_clean":"2024-05-27T07:00:53.776438+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T07:00:53.776438+0000","last_undegraded":"2024-05-27T07:00:53.776438+0000","last_fullsized":"2024-05-27T07:00:53.776438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647617+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T07:00:53.647617+0000","last_peered":"2024-05-27T07:00:53.647617+0000","last_clean":"2024-05-27T07:00:53.647617+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T07:00:53.647617+0000","last_undegraded":"2024-05-27T07:00:53.647617+0000","last_fullsized":"2024-05-27T07:00:53.647617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803766+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T07:00:53.803766+0000","last_peered":"2024-05-27T07:00:53.803766+0000","last_clean":"2024-05-27T07:00:53.803766+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T07:00:53.803766+0000","last_undegraded":"2024-05-27T07:00:53.803766+0000","last_fullsized":"2024-05-27T07:00:53.803766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203388+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T07:00:54.203388+0000","last_peered":"2024-05-27T07:00:54.203388+0000","last_clean":"2024-05-27T07:00:54.203388+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T07:00:54.203388+0000","last_undegraded":"2024-05-27T07:00:54.203388+0000","last_fullsized":"2024-05-27T07:00:54.203388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801019+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T07:00:53.801019+0000","last_peered":"2024-05-27T07:00:53.801019+0000","last_clean":"2024-05-27T07:00:53.801019+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T07:00:53.801019+0000","last_undegraded":"2024-05-27T07:00:53.801019+0000","last_fullsized":"2024-05-27T07:00:53.801019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844584+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T07:00:53.844584+0000","last_peered":"2024-05-27T07:00:53.844584+0000","last_clean":"2024-05-27T07:00:53.844584+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T07:00:53.844584+0000","last_undegraded":"2024-05-27T07:00:53.844584+0000","last_fullsized":"2024-05-27T07:00:53.844584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815380+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T07:00:53.815380+0000","last_peered":"2024-05-27T07:00:53.815380+0000","last_clean":"2024-05-27T07:00:53.815380+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T07:00:53.815380+0000","last_undegraded":"2024-05-27T07:00:53.815380+0000","last_fullsized":"2024-05-27T07:00:53.815380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.646214+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T07:00:53.646214+0000","last_peered":"2024-05-27T07:00:53.646214+0000","last_clean":"2024-05-27T07:00:53.646214+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T07:00:53.646214+0000","last_undegraded":"2024-05-27T07:00:53.646214+0000","last_fullsized":"2024-05-27T07:00:53.646214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801166+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T07:00:53.801166+0000","last_peered":"2024-05-27T07:00:53.801166+0000","last_clean":"2024-05-27T07:00:53.801166+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T07:00:53.801166+0000","last_undegraded":"2024-05-27T07:00:53.801166+0000","last_fullsized":"2024-05-27T07:00:53.801166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":565,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.250632+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T07:00:59.250632+0000","last_peered":"2024-05-27T07:00:59.250632+0000","last_clean":"2024-05-27T07:00:59.250632+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T07:00:59.250632+0000","last_undegraded":"2024-05-27T07:00:59.250632+0000","last_fullsized":"2024-05-27T07:00:59.250632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":510,"num_read_kb":510,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.652372+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T07:00:53.652372+0000","last_peered":"2024-05-27T07:00:53.652372+0000","last_clean":"2024-05-27T07:00:53.652372+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T07:00:53.652372+0000","last_undegraded":"2024-05-27T07:00:53.652372+0000","last_fullsized":"2024-05-27T07:00:53.652372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.645594+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T07:00:53.645594+0000","last_peered":"2024-05-27T07:00:53.645594+0000","last_clean":"2024-05-27T07:00:53.645594+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T07:00:53.645594+0000","last_undegraded":"2024-05-27T07:00:53.645594+0000","last_fullsized":"2024-05-27T07:00:53.645594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200781+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T07:00:54.200781+0000","last_peered":"2024-05-27T07:00:54.200781+0000","last_clean":"2024-05-27T07:00:54.200781+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T07:00:54.200781+0000","last_undegraded":"2024-05-27T07:00:54.200781+0000","last_fullsized":"2024-05-27T07:00:54.200781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844519+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T07:00:53.844519+0000","last_peered":"2024-05-27T07:00:53.844519+0000","last_clean":"2024-05-27T07:00:53.844519+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T07:00:53.844519+0000","last_undegraded":"2024-05-27T07:00:53.844519+0000","last_fullsized":"2024-05-27T07:00:53.844519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782729+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T07:00:53.782729+0000","last_peered":"2024-05-27T07:00:53.782729+0000","last_clean":"2024-05-27T07:00:53.782729+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T07:00:53.782729+0000","last_undegraded":"2024-05-27T07:00:53.782729+0000","last_fullsized":"2024-05-27T07:00:53.782729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816934+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T07:00:53.816934+0000","last_peered":"2024-05-27T07:00:53.816934+0000","last_clean":"2024-05-27T07:00:53.816934+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T07:00:53.816934+0000","last_undegraded":"2024-05-27T07:00:53.816934+0000","last_fullsized":"2024-05-27T07:00:53.816934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650831+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T07:00:53.650831+0000","last_peered":"2024-05-27T07:00:53.650831+0000","last_clean":"2024-05-27T07:00:53.650831+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T07:00:53.650831+0000","last_undegraded":"2024-05-27T07:00:53.650831+0000","last_fullsized":"2024-05-27T07:00:53.650831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647361+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T07:00:53.647361+0000","last_peered":"2024-05-27T07:00:53.647361+0000","last_clean":"2024-05-27T07:00:53.647361+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T07:00:53.647361+0000","last_undegraded":"2024-05-27T07:00:53.647361+0000","last_fullsized":"2024-05-27T07:00:53.647361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'46","reported_seq":636,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.269405+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T07:00:59.269405+0000","last_peered":"2024-05-27T07:00:59.269405+0000","last_clean":"2024-05-27T07:00:59.269405+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T07:00:59.269405+0000","last_undegraded":"2024-05-27T07:00:59.269405+0000","last_fullsized":"2024-05-27T07:00:59.269405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75230,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":526,"num_read_kb":336,"num_write":48,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791216+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T07:00:53.791216+0000","last_peered":"2024-05-27T07:00:53.791216+0000","last_clean":"2024-05-27T07:00:53.791216+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T07:00:53.791216+0000","last_undegraded":"2024-05-27T07:00:53.791216+0000","last_fullsized":"2024-05-27T07:00:53.791216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817811+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T07:00:53.817811+0000","last_peered":"2024-05-27T07:00:53.817811+0000","last_clean":"2024-05-27T07:00:53.817811+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T07:00:53.817811+0000","last_undegraded":"2024-05-27T07:00:53.817811+0000","last_fullsized":"2024-05-27T07:00:53.817811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650368+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T07:00:53.650368+0000","last_peered":"2024-05-27T07:00:53.650368+0000","last_clean":"2024-05-27T07:00:53.650368+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T07:00:53.650368+0000","last_undegraded":"2024-05-27T07:00:53.650368+0000","last_fullsized":"2024-05-27T07:00:53.650368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203525+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T07:00:54.203525+0000","last_peered":"2024-05-27T07:00:54.203525+0000","last_clean":"2024-05-27T07:00:54.203525+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T07:00:54.203525+0000","last_undegraded":"2024-05-27T07:00:54.203525+0000","last_fullsized":"2024-05-27T07:00:54.203525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803276+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T07:00:53.803276+0000","last_peered":"2024-05-27T07:00:53.803276+0000","last_clean":"2024-05-27T07:00:53.803276+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T07:00:53.803276+0000","last_undegraded":"2024-05-27T07:00:53.803276+0000","last_fullsized":"2024-05-27T07:00:53.803276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816169+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T07:00:53.816169+0000","last_peered":"2024-05-27T07:00:53.816169+0000","last_clean":"2024-05-27T07:00:53.816169+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T07:00:53.816169+0000","last_undegraded":"2024-05-27T07:00:53.816169+0000","last_fullsized":"2024-05-27T07:00:53.816169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200447+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T07:00:54.200447+0000","last_peered":"2024-05-27T07:00:54.200447+0000","last_clean":"2024-05-27T07:00:54.200447+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T07:00:54.200447+0000","last_undegraded":"2024-05-27T07:00:54.200447+0000","last_fullsized":"2024-05-27T07:00:54.200447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648912+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T07:00:53.648912+0000","last_peered":"2024-05-27T07:00:53.648912+0000","last_clean":"2024-05-27T07:00:53.648912+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T07:00:53.648912+0000","last_undegraded":"2024-05-27T07:00:53.648912+0000","last_fullsized":"2024-05-27T07:00:53.648912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266661+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T07:00:58.266661+0000","last_peered":"2024-05-27T07:00:58.266661+0000","last_clean":"2024-05-27T07:00:58.266661+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T07:00:58.266661+0000","last_undegraded":"2024-05-27T07:00:58.266661+0000","last_fullsized":"2024-05-27T07:00:58.266661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":8,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200387+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T07:00:54.200387+0000","last_peered":"2024-05-27T07:00:54.200387+0000","last_clean":"2024-05-27T07:00:54.200387+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T07:00:54.200387+0000","last_undegraded":"2024-05-27T07:00:54.200387+0000","last_fullsized":"2024-05-27T07:00:54.200387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204558+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T07:00:54.204558+0000","last_peered":"2024-05-27T07:00:54.204558+0000","last_clean":"2024-05-27T07:00:54.204558+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T07:00:54.204558+0000","last_undegraded":"2024-05-27T07:00:54.204558+0000","last_fullsized":"2024-05-27T07:00:54.204558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789874+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T07:00:53.789874+0000","last_peered":"2024-05-27T07:00:53.789874+0000","last_clean":"2024-05-27T07:00:53.789874+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T07:00:53.789874+0000","last_undegraded":"2024-05-27T07:00:53.789874+0000","last_fullsized":"2024-05-27T07:00:53.789874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815836+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T07:00:53.815836+0000","last_peered":"2024-05-27T07:00:53.815836+0000","last_clean":"2024-05-27T07:00:53.815836+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T07:00:53.815836+0000","last_undegraded":"2024-05-27T07:00:53.815836+0000","last_fullsized":"2024-05-27T07:00:53.815836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815656+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T07:00:53.815656+0000","last_peered":"2024-05-27T07:00:53.815656+0000","last_clean":"2024-05-27T07:00:53.815656+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T07:00:53.815656+0000","last_undegraded":"2024-05-27T07:00:53.815656+0000","last_fullsized":"2024-05-27T07:00:53.815656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792118+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T07:00:53.792118+0000","last_peered":"2024-05-27T07:00:53.792118+0000","last_clean":"2024-05-27T07:00:53.792118+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T07:00:53.792118+0000","last_undegraded":"2024-05-27T07:00:53.792118+0000","last_fullsized":"2024-05-27T07:00:53.792118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803956+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T07:00:53.803956+0000","last_peered":"2024-05-27T07:00:53.803956+0000","last_clean":"2024-05-27T07:00:53.803956+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T07:00:53.803956+0000","last_undegraded":"2024-05-27T07:00:53.803956+0000","last_fullsized":"2024-05-27T07:00:53.803956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790756+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T07:00:53.790756+0000","last_peered":"2024-05-27T07:00:53.790756+0000","last_clean":"2024-05-27T07:00:53.790756+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T07:00:53.790756+0000","last_undegraded":"2024-05-27T07:00:53.790756+0000","last_fullsized":"2024-05-27T07:00:53.790756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648046+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T07:00:53.648046+0000","last_peered":"2024-05-27T07:00:53.648046+0000","last_clean":"2024-05-27T07:00:53.648046+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T07:00:53.648046+0000","last_undegraded":"2024-05-27T07:00:53.648046+0000","last_fullsized":"2024-05-27T07:00:53.648046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776506+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T07:00:53.776506+0000","last_peered":"2024-05-27T07:00:53.776506+0000","last_clean":"2024-05-27T07:00:53.776506+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T07:00:53.776506+0000","last_undegraded":"2024-05-27T07:00:53.776506+0000","last_fullsized":"2024-05-27T07:00:53.776506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203958+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T07:00:54.203958+0000","last_peered":"2024-05-27T07:00:54.203958+0000","last_clean":"2024-05-27T07:00:54.203958+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T07:00:54.203958+0000","last_undegraded":"2024-05-27T07:00:54.203958+0000","last_fullsized":"2024-05-27T07:00:54.203958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'9","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.214416+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T07:00:58.214416+0000","last_peered":"2024-05-27T07:00:58.214416+0000","last_clean":"2024-05-27T07:00:58.214416+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T07:00:58.214416+0000","last_undegraded":"2024-05-27T07:00:58.214416+0000","last_fullsized":"2024-05-27T07:00:58.214416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776481+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T07:00:53.776481+0000","last_peered":"2024-05-27T07:00:53.776481+0000","last_clean":"2024-05-27T07:00:53.776481+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T07:00:53.776481+0000","last_undegraded":"2024-05-27T07:00:53.776481+0000","last_fullsized":"2024-05-27T07:00:53.776481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204687+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T07:00:54.204687+0000","last_peered":"2024-05-27T07:00:54.204687+0000","last_clean":"2024-05-27T07:00:54.204687+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T07:00:54.204687+0000","last_undegraded":"2024-05-27T07:00:54.204687+0000","last_fullsized":"2024-05-27T07:00:54.204687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815963+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T07:00:53.815963+0000","last_peered":"2024-05-27T07:00:53.815963+0000","last_clean":"2024-05-27T07:00:53.815963+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T07:00:53.815963+0000","last_undegraded":"2024-05-27T07:00:53.815963+0000","last_fullsized":"2024-05-27T07:00:53.815963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":311,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.264972+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T07:00:58.264972+0000","last_peered":"2024-05-27T07:00:58.264972+0000","last_clean":"2024-05-27T07:00:58.264972+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T07:00:58.264972+0000","last_undegraded":"2024-05-27T07:00:58.264972+0000","last_fullsized":"2024-05-27T07:00:58.264972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200533+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T07:00:54.200533+0000","last_peered":"2024-05-27T07:00:54.200533+0000","last_clean":"2024-05-27T07:00:54.200533+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T07:00:54.200533+0000","last_undegraded":"2024-05-27T07:00:54.200533+0000","last_fullsized":"2024-05-27T07:00:54.200533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801444+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T07:00:53.801444+0000","last_peered":"2024-05-27T07:00:53.801444+0000","last_clean":"2024-05-27T07:00:53.801444+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T07:00:53.801444+0000","last_undegraded":"2024-05-27T07:00:53.801444+0000","last_fullsized":"2024-05-27T07:00:53.801444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782485+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T07:00:53.782485+0000","last_peered":"2024-05-27T07:00:53.782485+0000","last_clean":"2024-05-27T07:00:53.782485+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T07:00:53.782485+0000","last_undegraded":"2024-05-27T07:00:53.782485+0000","last_fullsized":"2024-05-27T07:00:53.782485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:55.842641+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T07:00:55.842641+0000","last_peered":"2024-05-27T07:00:55.842641+0000","last_clean":"2024-05-27T07:00:55.842641+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T07:00:55.842641+0000","last_undegraded":"2024-05-27T07:00:55.842641+0000","last_fullsized":"2024-05-27T07:00:55.842641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816609+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T07:00:53.816609+0000","last_peered":"2024-05-27T07:00:53.816609+0000","last_clean":"2024-05-27T07:00:53.816609+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T07:00:53.816609+0000","last_undegraded":"2024-05-27T07:00:53.816609+0000","last_fullsized":"2024-05-27T07:00:53.816609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203145+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T07:00:54.203145+0000","last_peered":"2024-05-27T07:00:54.203145+0000","last_clean":"2024-05-27T07:00:54.203145+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T07:00:54.203145+0000","last_undegraded":"2024-05-27T07:00:54.203145+0000","last_fullsized":"2024-05-27T07:00:54.203145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200185+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T07:00:54.200185+0000","last_peered":"2024-05-27T07:00:54.200185+0000","last_clean":"2024-05-27T07:00:54.200185+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T07:00:54.200185+0000","last_undegraded":"2024-05-27T07:00:54.200185+0000","last_fullsized":"2024-05-27T07:00:54.200185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803223+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T07:00:53.803223+0000","last_peered":"2024-05-27T07:00:53.803223+0000","last_clean":"2024-05-27T07:00:53.803223+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T07:00:53.803223+0000","last_undegraded":"2024-05-27T07:00:53.803223+0000","last_fullsized":"2024-05-27T07:00:53.803223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791810+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T07:00:53.791810+0000","last_peered":"2024-05-27T07:00:53.791810+0000","last_clean":"2024-05-27T07:00:53.791810+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T07:00:53.791810+0000","last_undegraded":"2024-05-27T07:00:53.791810+0000","last_fullsized":"2024-05-27T07:00:53.791810+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204155+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T07:00:54.204155+0000","last_peered":"2024-05-27T07:00:54.204155+0000","last_clean":"2024-05-27T07:00:54.204155+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T07:00:54.204155+0000","last_undegraded":"2024-05-27T07:00:54.204155+0000","last_fullsized":"2024-05-27T07:00:54.204155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801014+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T07:00:53.801014+0000","last_peered":"2024-05-27T07:00:53.801014+0000","last_clean":"2024-05-27T07:00:53.801014+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T07:00:53.801014+0000","last_undegraded":"2024-05-27T07:00:53.801014+0000","last_fullsized":"2024-05-27T07:00:53.801014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'53","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.053173+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T07:00:54.053173+0000","last_peered":"2024-05-27T07:00:54.053173+0000","last_clean":"2024-05-27T07:00:54.053173+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T07:00:54.053173+0000","last_undegraded":"2024-05-27T07:00:54.053173+0000","last_fullsized":"2024-05-27T07:00:54.053173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":94,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802653+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T07:00:53.802653+0000","last_peered":"2024-05-27T07:00:53.802653+0000","last_clean":"2024-05-27T07:00:53.802653+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T07:00:53.802653+0000","last_undegraded":"2024-05-27T07:00:53.802653+0000","last_fullsized":"2024-05-27T07:00:53.802653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649305+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T07:00:53.649305+0000","last_peered":"2024-05-27T07:00:53.649305+0000","last_clean":"2024-05-27T07:00:53.649305+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T07:00:53.649305+0000","last_undegraded":"2024-05-27T07:00:53.649305+0000","last_fullsized":"2024-05-27T07:00:53.649305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783726+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T07:00:53.783726+0000","last_peered":"2024-05-27T07:00:53.783726+0000","last_clean":"2024-05-27T07:00:53.783726+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T07:00:53.783726+0000","last_undegraded":"2024-05-27T07:00:53.783726+0000","last_fullsized":"2024-05-27T07:00:53.783726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844741+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T07:00:53.844741+0000","last_peered":"2024-05-27T07:00:53.844741+0000","last_clean":"2024-05-27T07:00:53.844741+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T07:00:53.844741+0000","last_undegraded":"2024-05-27T07:00:53.844741+0000","last_fullsized":"2024-05-27T07:00:53.844741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783683+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T07:00:53.783683+0000","last_peered":"2024-05-27T07:00:53.783683+0000","last_clean":"2024-05-27T07:00:53.783683+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T07:00:53.783683+0000","last_undegraded":"2024-05-27T07:00:53.783683+0000","last_fullsized":"2024-05-27T07:00:53.783683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649444+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T07:00:53.649444+0000","last_peered":"2024-05-27T07:00:53.649444+0000","last_clean":"2024-05-27T07:00:53.649444+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T07:00:53.649444+0000","last_undegraded":"2024-05-27T07:00:53.649444+0000","last_fullsized":"2024-05-27T07:00:53.649444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789897+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T07:00:53.789897+0000","last_peered":"2024-05-27T07:00:53.789897+0000","last_clean":"2024-05-27T07:00:53.789897+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T07:00:53.789897+0000","last_undegraded":"2024-05-27T07:00:53.789897+0000","last_fullsized":"2024-05-27T07:00:53.789897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649098+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T07:00:53.649098+0000","last_peered":"2024-05-27T07:00:53.649098+0000","last_clean":"2024-05-27T07:00:53.649098+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T07:00:53.649098+0000","last_undegraded":"2024-05-27T07:00:53.649098+0000","last_fullsized":"2024-05-27T07:00:53.649098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801357+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T07:00:53.801357+0000","last_peered":"2024-05-27T07:00:53.801357+0000","last_clean":"2024-05-27T07:00:53.801357+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T07:00:53.801357+0000","last_undegraded":"2024-05-27T07:00:53.801357+0000","last_fullsized":"2024-05-27T07:00:53.801357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803270+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T07:00:53.803270+0000","last_peered":"2024-05-27T07:00:53.803270+0000","last_clean":"2024-05-27T07:00:53.803270+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T07:00:53.803270+0000","last_undegraded":"2024-05-27T07:00:53.803270+0000","last_fullsized":"2024-05-27T07:00:53.803270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783228+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T07:00:53.783228+0000","last_peered":"2024-05-27T07:00:53.783228+0000","last_clean":"2024-05-27T07:00:53.783228+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T07:00:53.783228+0000","last_undegraded":"2024-05-27T07:00:53.783228+0000","last_fullsized":"2024-05-27T07:00:53.783228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649930+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T07:00:53.649930+0000","last_peered":"2024-05-27T07:00:53.649930+0000","last_clean":"2024-05-27T07:00:53.649930+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T07:00:53.649930+0000","last_undegraded":"2024-05-27T07:00:53.649930+0000","last_fullsized":"2024-05-27T07:00:53.649930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801666+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T07:00:53.801666+0000","last_peered":"2024-05-27T07:00:53.801666+0000","last_clean":"2024-05-27T07:00:53.801666+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T07:00:53.801666+0000","last_undegraded":"2024-05-27T07:00:53.801666+0000","last_fullsized":"2024-05-27T07:00:53.801666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783466+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T07:00:53.783466+0000","last_peered":"2024-05-27T07:00:53.783466+0000","last_clean":"2024-05-27T07:00:53.783466+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T07:00:53.783466+0000","last_undegraded":"2024-05-27T07:00:53.783466+0000","last_fullsized":"2024-05-27T07:00:53.783466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790516+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T07:00:53.790516+0000","last_peered":"2024-05-27T07:00:53.790516+0000","last_clean":"2024-05-27T07:00:53.790516+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T07:00:53.790516+0000","last_undegraded":"2024-05-27T07:00:53.790516+0000","last_fullsized":"2024-05-27T07:00:53.790516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200234+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T07:00:54.200234+0000","last_peered":"2024-05-27T07:00:54.200234+0000","last_clean":"2024-05-27T07:00:54.200234+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T07:00:54.200234+0000","last_undegraded":"2024-05-27T07:00:54.200234+0000","last_fullsized":"2024-05-27T07:00:54.200234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649466+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T07:00:53.649466+0000","last_peered":"2024-05-27T07:00:53.649466+0000","last_clean":"2024-05-27T07:00:53.649466+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T07:00:53.649466+0000","last_undegraded":"2024-05-27T07:00:53.649466+0000","last_fullsized":"2024-05-27T07:00:53.649466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266640+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T07:00:58.266640+0000","last_peered":"2024-05-27T07:00:58.266640+0000","last_clean":"2024-05-27T07:00:58.266640+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T07:00:58.266640+0000","last_undegraded":"2024-05-27T07:00:58.266640+0000","last_fullsized":"2024-05-27T07:00:58.266640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":7,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200900+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T07:00:54.200900+0000","last_peered":"2024-05-27T07:00:54.200900+0000","last_clean":"2024-05-27T07:00:54.200900+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T07:00:54.200900+0000","last_undegraded":"2024-05-27T07:00:54.200900+0000","last_fullsized":"2024-05-27T07:00:54.200900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844028+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T07:00:53.844028+0000","last_peered":"2024-05-27T07:00:53.844028+0000","last_clean":"2024-05-27T07:00:53.844028+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T07:00:53.844028+0000","last_undegraded":"2024-05-27T07:00:53.844028+0000","last_fullsized":"2024-05-27T07:00:53.844028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.042149+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T07:00:54.042149+0000","last_peered":"2024-05-27T07:00:54.042149+0000","last_clean":"2024-05-27T07:00:54.042149+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T07:00:54.042149+0000","last_undegraded":"2024-05-27T07:00:54.042149+0000","last_fullsized":"2024-05-27T07:00:54.042149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200622+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T07:00:54.200622+0000","last_peered":"2024-05-27T07:00:54.200622+0000","last_clean":"2024-05-27T07:00:54.200622+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T07:00:54.200622+0000","last_undegraded":"2024-05-27T07:00:54.200622+0000","last_fullsized":"2024-05-27T07:00:54.200622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815204+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T07:00:53.815204+0000","last_peered":"2024-05-27T07:00:53.815204+0000","last_clean":"2024-05-27T07:00:53.815204+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T07:00:53.815204+0000","last_undegraded":"2024-05-27T07:00:53.815204+0000","last_fullsized":"2024-05-27T07:00:53.815204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204413+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T07:00:54.204413+0000","last_peered":"2024-05-27T07:00:54.204413+0000","last_clean":"2024-05-27T07:00:54.204413+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T07:00:54.204413+0000","last_undegraded":"2024-05-27T07:00:54.204413+0000","last_fullsized":"2024-05-27T07:00:54.204413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200845+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T07:00:54.200845+0000","last_peered":"2024-05-27T07:00:54.200845+0000","last_clean":"2024-05-27T07:00:54.200845+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T07:00:54.200845+0000","last_undegraded":"2024-05-27T07:00:54.200845+0000","last_fullsized":"2024-05-27T07:00:54.200845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803485+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T07:00:53.803485+0000","last_peered":"2024-05-27T07:00:53.803485+0000","last_clean":"2024-05-27T07:00:53.803485+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T07:00:53.803485+0000","last_undegraded":"2024-05-27T07:00:53.803485+0000","last_fullsized":"2024-05-27T07:00:53.803485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650051+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T07:00:53.650051+0000","last_peered":"2024-05-27T07:00:53.650051+0000","last_clean":"2024-05-27T07:00:53.650051+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T07:00:53.650051+0000","last_undegraded":"2024-05-27T07:00:53.650051+0000","last_fullsized":"2024-05-27T07:00:53.650051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649601+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T07:00:53.649601+0000","last_peered":"2024-05-27T07:00:53.649601+0000","last_clean":"2024-05-27T07:00:53.649601+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T07:00:53.649601+0000","last_undegraded":"2024-05-27T07:00:53.649601+0000","last_fullsized":"2024-05-27T07:00:53.649601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844301+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T07:00:53.844301+0000","last_peered":"2024-05-27T07:00:53.844301+0000","last_clean":"2024-05-27T07:00:53.844301+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T07:00:53.844301+0000","last_undegraded":"2024-05-27T07:00:53.844301+0000","last_fullsized":"2024-05-27T07:00:53.844301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204340+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T07:00:54.204340+0000","last_peered":"2024-05-27T07:00:54.204340+0000","last_clean":"2024-05-27T07:00:54.204340+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T07:00:54.204340+0000","last_undegraded":"2024-05-27T07:00:54.204340+0000","last_fullsized":"2024-05-27T07:00:54.204340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804068+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T07:00:53.804068+0000","last_peered":"2024-05-27T07:00:53.804068+0000","last_clean":"2024-05-27T07:00:53.804068+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T07:00:53.804068+0000","last_undegraded":"2024-05-27T07:00:53.804068+0000","last_fullsized":"2024-05-27T07:00:53.804068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651709+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T07:00:53.651709+0000","last_peered":"2024-05-27T07:00:53.651709+0000","last_clean":"2024-05-27T07:00:53.651709+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T07:00:53.651709+0000","last_undegraded":"2024-05-27T07:00:53.651709+0000","last_fullsized":"2024-05-27T07:00:53.651709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802724+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T07:00:53.802724+0000","last_peered":"2024-05-27T07:00:53.802724+0000","last_clean":"2024-05-27T07:00:53.802724+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T07:00:53.802724+0000","last_undegraded":"2024-05-27T07:00:53.802724+0000","last_fullsized":"2024-05-27T07:00:53.802724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203093+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T07:00:54.203093+0000","last_peered":"2024-05-27T07:00:54.203093+0000","last_clean":"2024-05-27T07:00:54.203093+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T07:00:54.203093+0000","last_undegraded":"2024-05-27T07:00:54.203093+0000","last_fullsized":"2024-05-27T07:00:54.203093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801509+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T07:00:53.801509+0000","last_peered":"2024-05-27T07:00:53.801509+0000","last_clean":"2024-05-27T07:00:53.801509+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T07:00:53.801509+0000","last_undegraded":"2024-05-27T07:00:53.801509+0000","last_fullsized":"2024-05-27T07:00:53.801509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650494+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T07:00:53.650494+0000","last_peered":"2024-05-27T07:00:53.650494+0000","last_clean":"2024-05-27T07:00:53.650494+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T07:00:53.650494+0000","last_undegraded":"2024-05-27T07:00:53.650494+0000","last_fullsized":"2024-05-27T07:00:53.650494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203784+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T07:00:54.203784+0000","last_peered":"2024-05-27T07:00:54.203784+0000","last_clean":"2024-05-27T07:00:54.203784+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T07:00:54.203784+0000","last_undegraded":"2024-05-27T07:00:54.203784+0000","last_fullsized":"2024-05-27T07:00:54.203784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803772+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T07:00:53.803772+0000","last_peered":"2024-05-27T07:00:53.803772+0000","last_clean":"2024-05-27T07:00:53.803772+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T07:00:53.803772+0000","last_undegraded":"2024-05-27T07:00:53.803772+0000","last_fullsized":"2024-05-27T07:00:53.803772+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612773,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2704,"num_read_kb":1481,"num_write":403,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1838319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678106,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1454080,"data_stored":925987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874337,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28876,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716276,"statfs":{"total":95995035648,"available":95965466624,"internally_reserved":0,"allocated":1904640,"data_stored":1326417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":928206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234095,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":531272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430328,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1679360,"data_stored":1131876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626559,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":527814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":604074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":529661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":530730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1050922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986318,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":607509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822791,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1255835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":396215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:01:03.049 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-27T07:01:03.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:01:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 -- 172.21.15.16:0/1677109361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10b670 msgr2=0x7f66cc10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 --2- 172.21.15.16:0/1677109361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10b670 0x7f66cc10da90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009900 tx=0x7f66b802a480 comp rx=0 tx=0).stop 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 -- 172.21.15.16:0/1677109361 shutdown_connections 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 --2- 172.21.15.16:0/1677109361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc10dfd0 0x7f66cc1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 --2- 172.21.15.16:0/1677109361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10b670 0x7f66cc10da90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 --2- 172.21.15.16:0/1677109361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc104c10 0x7f66cc105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 -- 172.21.15.16:0/1677109361 >> 172.21.15.16:0/1677109361 conn(0x7f66cc0780b0 msgr2=0x7f66cc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 -- 172.21.15.16:0/1677109361 shutdown_connections 2024-05-27T07:01:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.362+0000 7f66d0a00640 1 -- 172.21.15.16:0/1677109361 wait complete. 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.363+0000 7f66d0a00640 1 Processor -- start 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.363+0000 7f66d0a00640 1 -- start start 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.363+0000 7f66d0a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc104c10 0x7f66cc1056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.363+0000 7f66d0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc10b670 0x7f66cc105c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66d0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66d0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc113b10 con 0x7f66cc10dfd0 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66d0a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc113990 con 0x7f66cc104c10 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66d0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc113c90 con 0x7f66cc10b670 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37272/0 (socket says 172.21.15.16:37272) 2024-05-27T07:01:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 -- 172.21.15.16:0/3357592079 learned_addr learned my addr 172.21.15.16:0/3357592079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cb400640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc104c10 0x7f66cc1056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66caa00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc10b670 0x7f66cc105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 -- 172.21.15.16:0/3357592079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc10b670 msgr2=0x7f66cc105c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc10b670 0x7f66cc105c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 -- 172.21.15.16:0/3357592079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc104c10 msgr2=0x7f66cc1056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc104c10 0x7f66cc1056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.364+0000 7f66cbe00640 1 -- 172.21.15.16:0/3357592079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc1ace20 con 0x7f66cc10dfd0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66cbe00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f66c400eaa0 tx=0x7f66c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c400cdb0 con 0x7f66cc10dfd0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66c4005020 con 0x7f66cc10dfd0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc1acff0 con 0x7f66cc10dfd0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c40116d0 con 0x7f66cc10dfd0 2024-05-27T07:01:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.365+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1ad460 con 0x7f66cc10dfd0 2024-05-27T07:01:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.366+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66cc06a7d0 con 0x7f66cc10dfd0 2024-05-27T07:01:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.369+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f66c40051d0 con 0x7f66cc10dfd0 2024-05-27T07:01:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.370+0000 7f66c3e00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 0x7f66b00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.371+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f66c40e3240 con 0x7f66cc10dfd0 2024-05-27T07:01:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.371+0000 7f66cb400640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 0x7f66b00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.372+0000 7f66cb400640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 0x7f66b00b4f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005e10 tx=0x7f66bc005da0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.377+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66c4016030 con 0x7f66cc10dfd0 2024-05-27T07:01:03.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.590+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66cc10f4a0 con 0x7f66b00b2a70 2024-05-27T07:01:03.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.610+0000 7f66c3e00640 1 -- 172.21.15.16:0/3357592079 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339519 (secure 0 0 0) 0x7f66cc10f4a0 con 0x7f66b00b2a70 2024-05-27T07:01:03.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:03.609 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-27T07:01:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 msgr2=0x7f66b00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 0x7f66b00b4f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005e10 tx=0x7f66bc005da0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 msgr2=0x7f66cc106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f66c400eaa0 tx=0x7f66c400ef60 comp rx=0 tx=0).stop 2024-05-27T07:01:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 shutdown_connections 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f66b00b2a70 0x7f66b00b4f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc10dfd0 0x7f66cc106150 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66cc10b670 0x7f66cc105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 --2- 172.21.15.16:0/3357592079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66cc104c10 0x7f66cc1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.616+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 >> 172.21.15.16:0/3357592079 conn(0x7f66cc0780b0 msgr2=0x7f66cc10a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.617+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 shutdown_connections 2024-05-27T07:01:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:03.617+0000 7f66d0a00640 1 -- 172.21.15.16:0/3357592079 wait complete. 2024-05-27T07:01:03.634 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:01:01.677966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1203375,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2848,"num_read_kb":1729,"num_write":540,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":927,"ondisk_log_size":927,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386132,"kb_used_data":16160,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555692,"statfs":{"total":1151940427776,"available":1151545028608,"internally_reserved":0,"allocated":16547840,"data_stored":9950303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":210,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.968601"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804740+0000","last_change":"2024-05-27T06:55:10.381597+0000","last_active":"2024-05-27T07:00:53.804740+0000","last_peered":"2024-05-27T07:00:53.804740+0000","last_clean":"2024-05-27T07:00:53.804740+0000","last_became_active":"2024-05-27T06:55:10.381116+0000","last_became_peered":"2024-05-27T06:55:10.381116+0000","last_unstale":"2024-05-27T07:00:53.804740+0000","last_undegraded":"2024-05-27T07:00:53.804740+0000","last_fullsized":"2024-05-27T07:00:53.804740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:12.387426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844593+0000","last_change":"2024-05-27T06:55:10.370201+0000","last_active":"2024-05-27T07:00:53.844593+0000","last_peered":"2024-05-27T07:00:53.844593+0000","last_clean":"2024-05-27T07:00:53.844593+0000","last_became_active":"2024-05-27T06:55:10.369838+0000","last_became_peered":"2024-05-27T06:55:10.369838+0000","last_unstale":"2024-05-27T07:00:53.844593+0000","last_undegraded":"2024-05-27T07:00:53.844593+0000","last_fullsized":"2024-05-27T07:00:53.844593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.630808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844310+0000","last_change":"2024-05-27T06:55:10.357144+0000","last_active":"2024-05-27T07:00:53.844310+0000","last_peered":"2024-05-27T07:00:53.844310+0000","last_clean":"2024-05-27T07:00:53.844310+0000","last_became_active":"2024-05-27T06:55:10.356763+0000","last_became_peered":"2024-05-27T06:55:10.356763+0000","last_unstale":"2024-05-27T07:00:53.844310+0000","last_undegraded":"2024-05-27T07:00:53.844310+0000","last_fullsized":"2024-05-27T07:00:53.844310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:45.742966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782918+0000","last_change":"2024-05-27T06:55:10.347753+0000","last_active":"2024-05-27T07:00:53.782918+0000","last_peered":"2024-05-27T07:00:53.782918+0000","last_clean":"2024-05-27T07:00:53.782918+0000","last_became_active":"2024-05-27T06:55:10.346634+0000","last_became_peered":"2024-05-27T06:55:10.346634+0000","last_unstale":"2024-05-27T07:00:53.782918+0000","last_undegraded":"2024-05-27T07:00:53.782918+0000","last_fullsized":"2024-05-27T07:00:53.782918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:24.592204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790665+0000","last_change":"2024-05-27T06:55:10.350363+0000","last_active":"2024-05-27T07:00:53.790665+0000","last_peered":"2024-05-27T07:00:53.790665+0000","last_clean":"2024-05-27T07:00:53.790665+0000","last_became_active":"2024-05-27T06:55:10.349817+0000","last_became_peered":"2024-05-27T06:55:10.349817+0000","last_unstale":"2024-05-27T07:00:53.790665+0000","last_undegraded":"2024-05-27T07:00:53.790665+0000","last_fullsized":"2024-05-27T07:00:53.790665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:23.527203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776643+0000","last_change":"2024-05-27T06:55:39.330849+0000","last_active":"2024-05-27T07:00:53.776643+0000","last_peered":"2024-05-27T07:00:53.776643+0000","last_clean":"2024-05-27T07:00:53.776643+0000","last_became_active":"2024-05-27T06:55:39.330075+0000","last_became_peered":"2024-05-27T06:55:39.330075+0000","last_unstale":"2024-05-27T07:00:53.776643+0000","last_undegraded":"2024-05-27T07:00:53.776643+0000","last_fullsized":"2024-05-27T07:00:53.776643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:49.164322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200158+0000","last_change":"2024-05-27T06:55:10.369462+0000","last_active":"2024-05-27T07:00:54.200158+0000","last_peered":"2024-05-27T07:00:54.200158+0000","last_clean":"2024-05-27T07:00:54.200158+0000","last_became_active":"2024-05-27T06:55:10.369073+0000","last_became_peered":"2024-05-27T06:55:10.369073+0000","last_unstale":"2024-05-27T07:00:54.200158+0000","last_undegraded":"2024-05-27T07:00:54.200158+0000","last_fullsized":"2024-05-27T07:00:54.200158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:06.461354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801179+0000","last_change":"2024-05-27T06:55:10.373423+0000","last_active":"2024-05-27T07:00:53.801179+0000","last_peered":"2024-05-27T07:00:53.801179+0000","last_clean":"2024-05-27T07:00:53.801179+0000","last_became_active":"2024-05-27T06:55:10.369991+0000","last_became_peered":"2024-05-27T06:55:10.369991+0000","last_unstale":"2024-05-27T07:00:53.801179+0000","last_undegraded":"2024-05-27T07:00:53.801179+0000","last_fullsized":"2024-05-27T07:00:53.801179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:22.441006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651181+0000","last_change":"2024-05-27T06:55:10.381642+0000","last_active":"2024-05-27T07:00:53.651181+0000","last_peered":"2024-05-27T07:00:53.651181+0000","last_clean":"2024-05-27T07:00:53.651181+0000","last_became_active":"2024-05-27T06:55:10.381240+0000","last_became_peered":"2024-05-27T06:55:10.381240+0000","last_unstale":"2024-05-27T07:00:53.651181+0000","last_undegraded":"2024-05-27T07:00:53.651181+0000","last_fullsized":"2024-05-27T07:00:53.651181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:17.920587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.777439+0000","last_change":"2024-05-27T06:55:39.334537+0000","last_active":"2024-05-27T07:00:53.777439+0000","last_peered":"2024-05-27T07:00:53.777439+0000","last_clean":"2024-05-27T07:00:53.777439+0000","last_became_active":"2024-05-27T06:55:39.333080+0000","last_became_peered":"2024-05-27T06:55:39.333080+0000","last_unstale":"2024-05-27T07:00:53.777439+0000","last_undegraded":"2024-05-27T07:00:53.777439+0000","last_fullsized":"2024-05-27T07:00:53.777439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:14:58.628084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200612+0000","last_change":"2024-05-27T06:55:10.366792+0000","last_active":"2024-05-27T07:00:54.200612+0000","last_peered":"2024-05-27T07:00:54.200612+0000","last_clean":"2024-05-27T07:00:54.200612+0000","last_became_active":"2024-05-27T06:55:10.366392+0000","last_became_peered":"2024-05-27T06:55:10.366392+0000","last_unstale":"2024-05-27T07:00:54.200612+0000","last_undegraded":"2024-05-27T07:00:54.200612+0000","last_fullsized":"2024-05-27T07:00:54.200612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:54.873760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783009+0000","last_change":"2024-05-27T06:55:10.369326+0000","last_active":"2024-05-27T07:00:53.783009+0000","last_peered":"2024-05-27T07:00:53.783009+0000","last_clean":"2024-05-27T07:00:53.783009+0000","last_became_active":"2024-05-27T06:55:10.367919+0000","last_became_peered":"2024-05-27T06:55:10.367919+0000","last_unstale":"2024-05-27T07:00:53.783009+0000","last_undegraded":"2024-05-27T07:00:53.783009+0000","last_fullsized":"2024-05-27T07:00:53.783009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:19.975149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203914+0000","last_change":"2024-05-27T06:55:10.374236+0000","last_active":"2024-05-27T07:00:54.203914+0000","last_peered":"2024-05-27T07:00:54.203914+0000","last_clean":"2024-05-27T07:00:54.203914+0000","last_became_active":"2024-05-27T06:55:10.373870+0000","last_became_peered":"2024-05-27T06:55:10.373870+0000","last_unstale":"2024-05-27T07:00:54.203914+0000","last_undegraded":"2024-05-27T07:00:54.203914+0000","last_fullsized":"2024-05-27T07:00:54.203914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:42.829455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843989+0000","last_change":"2024-05-27T06:55:10.372019+0000","last_active":"2024-05-27T07:00:53.843989+0000","last_peered":"2024-05-27T07:00:53.843989+0000","last_clean":"2024-05-27T07:00:53.843989+0000","last_became_active":"2024-05-27T06:55:10.371610+0000","last_became_peered":"2024-05-27T06:55:10.371610+0000","last_unstale":"2024-05-27T07:00:53.843989+0000","last_undegraded":"2024-05-27T07:00:53.843989+0000","last_fullsized":"2024-05-27T07:00:53.843989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:02.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.843868+0000","last_change":"2024-05-27T06:55:10.355285+0000","last_active":"2024-05-27T07:00:53.843868+0000","last_peered":"2024-05-27T07:00:53.843868+0000","last_clean":"2024-05-27T07:00:53.843868+0000","last_became_active":"2024-05-27T06:55:10.352086+0000","last_became_peered":"2024-05-27T06:55:10.352086+0000","last_unstale":"2024-05-27T07:00:53.843868+0000","last_undegraded":"2024-05-27T07:00:53.843868+0000","last_fullsized":"2024-05-27T07:00:53.843868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:04.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649192+0000","last_change":"2024-05-27T06:55:10.362321+0000","last_active":"2024-05-27T07:00:53.649192+0000","last_peered":"2024-05-27T07:00:53.649192+0000","last_clean":"2024-05-27T07:00:53.649192+0000","last_became_active":"2024-05-27T06:55:10.361823+0000","last_became_peered":"2024-05-27T06:55:10.361823+0000","last_unstale":"2024-05-27T07:00:53.649192+0000","last_undegraded":"2024-05-27T07:00:53.649192+0000","last_fullsized":"2024-05-27T07:00:53.649192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:14:32.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804415+0000","last_change":"2024-05-27T06:55:10.382575+0000","last_active":"2024-05-27T07:00:53.804415+0000","last_peered":"2024-05-27T07:00:53.804415+0000","last_clean":"2024-05-27T07:00:53.804415+0000","last_became_active":"2024-05-27T06:55:10.382178+0000","last_became_peered":"2024-05-27T06:55:10.382178+0000","last_unstale":"2024-05-27T07:00:53.804415+0000","last_undegraded":"2024-05-27T07:00:53.804415+0000","last_fullsized":"2024-05-27T07:00:53.804415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:17:59.298752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648444+0000","last_change":"2024-05-27T06:55:10.372299+0000","last_active":"2024-05-27T07:00:53.648444+0000","last_peered":"2024-05-27T07:00:53.648444+0000","last_clean":"2024-05-27T07:00:53.648444+0000","last_became_active":"2024-05-27T06:55:10.371186+0000","last_became_peered":"2024-05-27T06:55:10.371186+0000","last_unstale":"2024-05-27T07:00:53.648444+0000","last_undegraded":"2024-05-27T07:00:53.648444+0000","last_fullsized":"2024-05-27T07:00:53.648444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:24:39.877648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651138+0000","last_change":"2024-05-27T06:55:39.332134+0000","last_active":"2024-05-27T07:00:53.651138+0000","last_peered":"2024-05-27T07:00:53.651138+0000","last_clean":"2024-05-27T07:00:53.651138+0000","last_became_active":"2024-05-27T06:55:39.330362+0000","last_became_peered":"2024-05-27T06:55:39.330362+0000","last_unstale":"2024-05-27T07:00:53.651138+0000","last_undegraded":"2024-05-27T07:00:53.651138+0000","last_fullsized":"2024-05-27T07:00:53.651138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:40:11.836759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649738+0000","last_change":"2024-05-27T06:56:40.410046+0000","last_active":"2024-05-27T07:00:53.649738+0000","last_peered":"2024-05-27T07:00:53.649738+0000","last_clean":"2024-05-27T07:00:53.649738+0000","last_became_active":"2024-05-27T06:56:40.409022+0000","last_became_peered":"2024-05-27T06:56:40.409022+0000","last_unstale":"2024-05-27T07:00:53.649738+0000","last_undegraded":"2024-05-27T07:00:53.649738+0000","last_fullsized":"2024-05-27T07:00:53.649738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:33.408146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":319,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.213065+0000","last_change":"2024-05-27T06:55:05.298911+0000","last_active":"2024-05-27T07:00:58.213065+0000","last_peered":"2024-05-27T07:00:58.213065+0000","last_clean":"2024-05-27T07:00:58.213065+0000","last_became_active":"2024-05-27T06:55:05.298350+0000","last_became_peered":"2024-05-27T06:55:05.298350+0000","last_unstale":"2024-05-27T07:00:58.213065+0000","last_undegraded":"2024-05-27T07:00:58.213065+0000","last_fullsized":"2024-05-27T07:00:58.213065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:56.999193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803028+0000","last_change":"2024-05-27T06:55:10.375654+0000","last_active":"2024-05-27T07:00:53.803028+0000","last_peered":"2024-05-27T07:00:53.803028+0000","last_clean":"2024-05-27T07:00:53.803028+0000","last_became_active":"2024-05-27T06:55:10.375299+0000","last_became_peered":"2024-05-27T06:55:10.375299+0000","last_unstale":"2024-05-27T07:00:53.803028+0000","last_undegraded":"2024-05-27T07:00:53.803028+0000","last_fullsized":"2024-05-27T07:00:53.803028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:25.940831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.781717+0000","last_change":"2024-05-27T06:56:40.407895+0000","last_active":"2024-05-27T07:00:53.781717+0000","last_peered":"2024-05-27T07:00:53.781717+0000","last_clean":"2024-05-27T07:00:53.781717+0000","last_became_active":"2024-05-27T06:56:40.407417+0000","last_became_peered":"2024-05-27T06:56:40.407417+0000","last_unstale":"2024-05-27T07:00:53.781717+0000","last_undegraded":"2024-05-27T07:00:53.781717+0000","last_fullsized":"2024-05-27T07:00:53.781717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:10:53.270210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803020+0000","last_change":"2024-05-27T06:55:05.309086+0000","last_active":"2024-05-27T07:00:53.803020+0000","last_peered":"2024-05-27T07:00:53.803020+0000","last_clean":"2024-05-27T07:00:53.803020+0000","last_became_active":"2024-05-27T06:55:05.307788+0000","last_became_peered":"2024-05-27T06:55:05.307788+0000","last_unstale":"2024-05-27T07:00:53.803020+0000","last_undegraded":"2024-05-27T07:00:53.803020+0000","last_fullsized":"2024-05-27T07:00:53.803020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:45.219442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844590+0000","last_change":"2024-05-27T06:55:10.358471+0000","last_active":"2024-05-27T07:00:53.844590+0000","last_peered":"2024-05-27T07:00:53.844590+0000","last_clean":"2024-05-27T07:00:53.844590+0000","last_became_active":"2024-05-27T06:55:10.358055+0000","last_became_peered":"2024-05-27T06:55:10.358055+0000","last_unstale":"2024-05-27T07:00:53.844590+0000","last_undegraded":"2024-05-27T07:00:53.844590+0000","last_fullsized":"2024-05-27T07:00:53.844590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:19.119047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782289+0000","last_change":"2024-05-27T06:55:26.229286+0000","last_active":"2024-05-27T07:00:53.782289+0000","last_peered":"2024-05-27T07:00:53.782289+0000","last_clean":"2024-05-27T07:00:53.782289+0000","last_became_active":"2024-05-27T06:55:26.228740+0000","last_became_peered":"2024-05-27T06:55:26.228740+0000","last_unstale":"2024-05-27T07:00:53.782289+0000","last_undegraded":"2024-05-27T07:00:53.782289+0000","last_fullsized":"2024-05-27T07:00:53.782289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:50.475691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792226+0000","last_change":"2024-05-27T06:55:05.298122+0000","last_active":"2024-05-27T07:00:53.792226+0000","last_peered":"2024-05-27T07:00:53.792226+0000","last_clean":"2024-05-27T07:00:53.792226+0000","last_became_active":"2024-05-27T06:55:05.297557+0000","last_became_peered":"2024-05-27T06:55:05.297557+0000","last_unstale":"2024-05-27T07:00:53.792226+0000","last_undegraded":"2024-05-27T07:00:53.792226+0000","last_fullsized":"2024-05-27T07:00:53.792226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:19.602673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650377+0000","last_change":"2024-05-27T06:55:10.368726+0000","last_active":"2024-05-27T07:00:53.650377+0000","last_peered":"2024-05-27T07:00:53.650377+0000","last_clean":"2024-05-27T07:00:53.650377+0000","last_became_active":"2024-05-27T06:55:10.368309+0000","last_became_peered":"2024-05-27T06:55:10.368309+0000","last_unstale":"2024-05-27T07:00:53.650377+0000","last_undegraded":"2024-05-27T07:00:53.650377+0000","last_fullsized":"2024-05-27T07:00:53.650377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:27.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782034+0000","last_change":"2024-05-27T06:56:40.407803+0000","last_active":"2024-05-27T07:00:53.782034+0000","last_peered":"2024-05-27T07:00:53.782034+0000","last_clean":"2024-05-27T07:00:53.782034+0000","last_became_active":"2024-05-27T06:56:40.407083+0000","last_became_peered":"2024-05-27T06:56:40.407083+0000","last_unstale":"2024-05-27T07:00:53.782034+0000","last_undegraded":"2024-05-27T07:00:53.782034+0000","last_fullsized":"2024-05-27T07:00:53.782034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:23.379611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791486+0000","last_change":"2024-05-27T06:55:05.301624+0000","last_active":"2024-05-27T07:00:53.791486+0000","last_peered":"2024-05-27T07:00:53.791486+0000","last_clean":"2024-05-27T07:00:53.791486+0000","last_became_active":"2024-05-27T06:55:05.301178+0000","last_became_peered":"2024-05-27T06:55:05.301178+0000","last_unstale":"2024-05-27T07:00:53.791486+0000","last_undegraded":"2024-05-27T07:00:53.791486+0000","last_fullsized":"2024-05-27T07:00:53.791486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:39:01.933057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817046+0000","last_change":"2024-05-27T06:55:10.373605+0000","last_active":"2024-05-27T07:00:53.817046+0000","last_peered":"2024-05-27T07:00:53.817046+0000","last_clean":"2024-05-27T07:00:53.817046+0000","last_became_active":"2024-05-27T06:55:10.372178+0000","last_became_peered":"2024-05-27T06:55:10.372178+0000","last_unstale":"2024-05-27T07:00:53.817046+0000","last_undegraded":"2024-05-27T07:00:53.817046+0000","last_fullsized":"2024-05-27T07:00:53.817046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:47.969042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650642+0000","last_change":"2024-05-27T06:55:26.236126+0000","last_active":"2024-05-27T07:00:53.650642+0000","last_peered":"2024-05-27T07:00:53.650642+0000","last_clean":"2024-05-27T07:00:53.650642+0000","last_became_active":"2024-05-27T06:55:26.234333+0000","last_became_peered":"2024-05-27T06:55:26.234333+0000","last_unstale":"2024-05-27T07:00:53.650642+0000","last_undegraded":"2024-05-27T07:00:53.650642+0000","last_fullsized":"2024-05-27T07:00:53.650642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:25:31.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":315,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.248852+0000","last_change":"2024-05-27T06:55:05.299920+0000","last_active":"2024-05-27T07:00:58.248852+0000","last_peered":"2024-05-27T07:00:58.248852+0000","last_clean":"2024-05-27T07:00:58.248852+0000","last_became_active":"2024-05-27T06:55:05.299386+0000","last_became_peered":"2024-05-27T06:55:05.299386+0000","last_unstale":"2024-05-27T07:00:58.248852+0000","last_undegraded":"2024-05-27T07:00:58.248852+0000","last_fullsized":"2024-05-27T07:00:58.248852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:59:51.312477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":259,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790375+0000","last_change":"2024-05-27T06:55:10.368144+0000","last_active":"2024-05-27T07:00:53.790375+0000","last_peered":"2024-05-27T07:00:53.790375+0000","last_clean":"2024-05-27T07:00:53.790375+0000","last_became_active":"2024-05-27T06:55:10.367731+0000","last_became_peered":"2024-05-27T06:55:10.367731+0000","last_unstale":"2024-05-27T07:00:53.790375+0000","last_undegraded":"2024-05-27T07:00:53.790375+0000","last_fullsized":"2024-05-27T07:00:53.790375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:14.943554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650611+0000","last_change":"2024-05-27T06:55:26.229377+0000","last_active":"2024-05-27T07:00:53.650611+0000","last_peered":"2024-05-27T07:00:53.650611+0000","last_clean":"2024-05-27T07:00:53.650611+0000","last_became_active":"2024-05-27T06:55:26.228921+0000","last_became_peered":"2024-05-27T06:55:26.228921+0000","last_unstale":"2024-05-27T07:00:53.650611+0000","last_undegraded":"2024-05-27T07:00:53.650611+0000","last_fullsized":"2024-05-27T07:00:53.650611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:56:26.522054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776438+0000","last_change":"2024-05-27T06:55:05.300549+0000","last_active":"2024-05-27T07:00:53.776438+0000","last_peered":"2024-05-27T07:00:53.776438+0000","last_clean":"2024-05-27T07:00:53.776438+0000","last_became_active":"2024-05-27T06:55:05.300156+0000","last_became_peered":"2024-05-27T06:55:05.300156+0000","last_unstale":"2024-05-27T07:00:53.776438+0000","last_undegraded":"2024-05-27T07:00:53.776438+0000","last_fullsized":"2024-05-27T07:00:53.776438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:19.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647617+0000","last_change":"2024-05-27T06:55:39.331939+0000","last_active":"2024-05-27T07:00:53.647617+0000","last_peered":"2024-05-27T07:00:53.647617+0000","last_clean":"2024-05-27T07:00:53.647617+0000","last_became_active":"2024-05-27T06:55:39.331334+0000","last_became_peered":"2024-05-27T06:55:39.331334+0000","last_unstale":"2024-05-27T07:00:53.647617+0000","last_undegraded":"2024-05-27T07:00:53.647617+0000","last_fullsized":"2024-05-27T07:00:53.647617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:35.757908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803766+0000","last_change":"2024-05-27T06:55:26.237999+0000","last_active":"2024-05-27T07:00:53.803766+0000","last_peered":"2024-05-27T07:00:53.803766+0000","last_clean":"2024-05-27T07:00:53.803766+0000","last_became_active":"2024-05-27T06:55:26.237229+0000","last_became_peered":"2024-05-27T06:55:26.237229+0000","last_unstale":"2024-05-27T07:00:53.803766+0000","last_undegraded":"2024-05-27T07:00:53.803766+0000","last_fullsized":"2024-05-27T07:00:53.803766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:20:58.593313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203388+0000","last_change":"2024-05-27T06:55:05.292802+0000","last_active":"2024-05-27T07:00:54.203388+0000","last_peered":"2024-05-27T07:00:54.203388+0000","last_clean":"2024-05-27T07:00:54.203388+0000","last_became_active":"2024-05-27T06:55:05.292377+0000","last_became_peered":"2024-05-27T06:55:05.292377+0000","last_unstale":"2024-05-27T07:00:54.203388+0000","last_undegraded":"2024-05-27T07:00:54.203388+0000","last_fullsized":"2024-05-27T07:00:54.203388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:38.243915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801019+0000","last_change":"2024-05-27T06:55:10.373879+0000","last_active":"2024-05-27T07:00:53.801019+0000","last_peered":"2024-05-27T07:00:53.801019+0000","last_clean":"2024-05-27T07:00:53.801019+0000","last_became_active":"2024-05-27T06:55:10.373480+0000","last_became_peered":"2024-05-27T06:55:10.373480+0000","last_unstale":"2024-05-27T07:00:53.801019+0000","last_undegraded":"2024-05-27T07:00:53.801019+0000","last_fullsized":"2024-05-27T07:00:53.801019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:44:28.672116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844584+0000","last_change":"2024-05-27T06:55:26.224021+0000","last_active":"2024-05-27T07:00:53.844584+0000","last_peered":"2024-05-27T07:00:53.844584+0000","last_clean":"2024-05-27T07:00:53.844584+0000","last_became_active":"2024-05-27T06:55:26.223237+0000","last_became_peered":"2024-05-27T06:55:26.223237+0000","last_unstale":"2024-05-27T07:00:53.844584+0000","last_undegraded":"2024-05-27T07:00:53.844584+0000","last_fullsized":"2024-05-27T07:00:53.844584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:23.415000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815380+0000","last_change":"2024-05-27T06:55:05.306305+0000","last_active":"2024-05-27T07:00:53.815380+0000","last_peered":"2024-05-27T07:00:53.815380+0000","last_clean":"2024-05-27T07:00:53.815380+0000","last_became_active":"2024-05-27T06:55:05.305785+0000","last_became_peered":"2024-05-27T06:55:05.305785+0000","last_unstale":"2024-05-27T07:00:53.815380+0000","last_undegraded":"2024-05-27T07:00:53.815380+0000","last_fullsized":"2024-05-27T07:00:53.815380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:29:42.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.646214+0000","last_change":"2024-05-27T06:55:10.360898+0000","last_active":"2024-05-27T07:00:53.646214+0000","last_peered":"2024-05-27T07:00:53.646214+0000","last_clean":"2024-05-27T07:00:53.646214+0000","last_became_active":"2024-05-27T06:55:10.360414+0000","last_became_peered":"2024-05-27T06:55:10.360414+0000","last_unstale":"2024-05-27T07:00:53.646214+0000","last_undegraded":"2024-05-27T07:00:53.646214+0000","last_fullsized":"2024-05-27T07:00:53.646214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.870591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801166+0000","last_change":"2024-05-27T06:55:26.235868+0000","last_active":"2024-05-27T07:00:53.801166+0000","last_peered":"2024-05-27T07:00:53.801166+0000","last_clean":"2024-05-27T07:00:53.801166+0000","last_became_active":"2024-05-27T06:55:26.233491+0000","last_became_peered":"2024-05-27T06:55:26.233491+0000","last_unstale":"2024-05-27T07:00:53.801166+0000","last_undegraded":"2024-05-27T07:00:53.801166+0000","last_fullsized":"2024-05-27T07:00:53.801166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:16:10.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":565,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.250632+0000","last_change":"2024-05-27T06:55:05.301180+0000","last_active":"2024-05-27T07:00:59.250632+0000","last_peered":"2024-05-27T07:00:59.250632+0000","last_clean":"2024-05-27T07:00:59.250632+0000","last_became_active":"2024-05-27T06:55:05.300329+0000","last_became_peered":"2024-05-27T06:55:05.300329+0000","last_unstale":"2024-05-27T07:00:59.250632+0000","last_undegraded":"2024-05-27T07:00:59.250632+0000","last_fullsized":"2024-05-27T07:00:59.250632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:18.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":510,"num_read_kb":510,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.652372+0000","last_change":"2024-05-27T06:55:10.373059+0000","last_active":"2024-05-27T07:00:53.652372+0000","last_peered":"2024-05-27T07:00:53.652372+0000","last_clean":"2024-05-27T07:00:53.652372+0000","last_became_active":"2024-05-27T06:55:10.372640+0000","last_became_peered":"2024-05-27T06:55:10.372640+0000","last_unstale":"2024-05-27T07:00:53.652372+0000","last_undegraded":"2024-05-27T07:00:53.652372+0000","last_fullsized":"2024-05-27T07:00:53.652372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:04.237872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.645594+0000","last_change":"2024-05-27T06:55:26.213022+0000","last_active":"2024-05-27T07:00:53.645594+0000","last_peered":"2024-05-27T07:00:53.645594+0000","last_clean":"2024-05-27T07:00:53.645594+0000","last_became_active":"2024-05-27T06:55:26.212459+0000","last_became_peered":"2024-05-27T06:55:26.212459+0000","last_unstale":"2024-05-27T07:00:53.645594+0000","last_undegraded":"2024-05-27T07:00:53.645594+0000","last_fullsized":"2024-05-27T07:00:53.645594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:47.659486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200781+0000","last_change":"2024-05-27T06:55:05.292502+0000","last_active":"2024-05-27T07:00:54.200781+0000","last_peered":"2024-05-27T07:00:54.200781+0000","last_clean":"2024-05-27T07:00:54.200781+0000","last_became_active":"2024-05-27T06:55:05.291587+0000","last_became_peered":"2024-05-27T06:55:05.291587+0000","last_unstale":"2024-05-27T07:00:54.200781+0000","last_undegraded":"2024-05-27T07:00:54.200781+0000","last_fullsized":"2024-05-27T07:00:54.200781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:47.582110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844519+0000","last_change":"2024-05-27T06:55:10.365755+0000","last_active":"2024-05-27T07:00:53.844519+0000","last_peered":"2024-05-27T07:00:53.844519+0000","last_clean":"2024-05-27T07:00:53.844519+0000","last_became_active":"2024-05-27T06:55:10.365364+0000","last_became_peered":"2024-05-27T06:55:10.365364+0000","last_unstale":"2024-05-27T07:00:53.844519+0000","last_undegraded":"2024-05-27T07:00:53.844519+0000","last_fullsized":"2024-05-27T07:00:53.844519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:41.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782729+0000","last_change":"2024-05-27T06:55:39.333470+0000","last_active":"2024-05-27T07:00:53.782729+0000","last_peered":"2024-05-27T07:00:53.782729+0000","last_clean":"2024-05-27T07:00:53.782729+0000","last_became_active":"2024-05-27T06:55:39.332262+0000","last_became_peered":"2024-05-27T06:55:39.332262+0000","last_unstale":"2024-05-27T07:00:53.782729+0000","last_undegraded":"2024-05-27T07:00:53.782729+0000","last_fullsized":"2024-05-27T07:00:53.782729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:30:55.850806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816934+0000","last_change":"2024-05-27T06:55:05.311054+0000","last_active":"2024-05-27T07:00:53.816934+0000","last_peered":"2024-05-27T07:00:53.816934+0000","last_clean":"2024-05-27T07:00:53.816934+0000","last_became_active":"2024-05-27T06:55:05.310573+0000","last_became_peered":"2024-05-27T06:55:05.310573+0000","last_unstale":"2024-05-27T07:00:53.816934+0000","last_undegraded":"2024-05-27T07:00:53.816934+0000","last_fullsized":"2024-05-27T07:00:53.816934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:33.432507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650831+0000","last_change":"2024-05-27T06:55:10.359860+0000","last_active":"2024-05-27T07:00:53.650831+0000","last_peered":"2024-05-27T07:00:53.650831+0000","last_clean":"2024-05-27T07:00:53.650831+0000","last_became_active":"2024-05-27T06:55:10.359026+0000","last_became_peered":"2024-05-27T06:55:10.359026+0000","last_unstale":"2024-05-27T07:00:53.650831+0000","last_undegraded":"2024-05-27T07:00:53.650831+0000","last_fullsized":"2024-05-27T07:00:53.650831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:57:44.735162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.647361+0000","last_change":"2024-05-27T06:55:26.232229+0000","last_active":"2024-05-27T07:00:53.647361+0000","last_peered":"2024-05-27T07:00:53.647361+0000","last_clean":"2024-05-27T07:00:53.647361+0000","last_became_active":"2024-05-27T06:55:26.229280+0000","last_became_peered":"2024-05-27T06:55:26.229280+0000","last_unstale":"2024-05-27T07:00:53.647361+0000","last_undegraded":"2024-05-27T07:00:53.647361+0000","last_fullsized":"2024-05-27T07:00:53.647361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:53.964363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'46","reported_seq":636,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:59.269405+0000","last_change":"2024-05-27T06:55:39.351879+0000","last_active":"2024-05-27T07:00:59.269405+0000","last_peered":"2024-05-27T07:00:59.269405+0000","last_clean":"2024-05-27T07:00:59.269405+0000","last_became_active":"2024-05-27T06:55:39.331116+0000","last_became_peered":"2024-05-27T06:55:39.331116+0000","last_unstale":"2024-05-27T07:00:59.269405+0000","last_undegraded":"2024-05-27T07:00:59.269405+0000","last_fullsized":"2024-05-27T07:00:59.269405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:08:01.265154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75230,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":526,"num_read_kb":336,"num_write":48,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791216+0000","last_change":"2024-05-27T06:55:10.362755+0000","last_active":"2024-05-27T07:00:53.791216+0000","last_peered":"2024-05-27T07:00:53.791216+0000","last_clean":"2024-05-27T07:00:53.791216+0000","last_became_active":"2024-05-27T06:55:10.361055+0000","last_became_peered":"2024-05-27T06:55:10.361055+0000","last_unstale":"2024-05-27T07:00:53.791216+0000","last_undegraded":"2024-05-27T07:00:53.791216+0000","last_fullsized":"2024-05-27T07:00:53.791216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:02.406375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.817811+0000","last_change":"2024-05-27T06:55:26.234899+0000","last_active":"2024-05-27T07:00:53.817811+0000","last_peered":"2024-05-27T07:00:53.817811+0000","last_clean":"2024-05-27T07:00:53.817811+0000","last_became_active":"2024-05-27T06:55:26.233689+0000","last_became_peered":"2024-05-27T06:55:26.233689+0000","last_unstale":"2024-05-27T07:00:53.817811+0000","last_undegraded":"2024-05-27T07:00:53.817811+0000","last_fullsized":"2024-05-27T07:00:53.817811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:45.862402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650368+0000","last_change":"2024-05-27T06:55:05.301171+0000","last_active":"2024-05-27T07:00:53.650368+0000","last_peered":"2024-05-27T07:00:53.650368+0000","last_clean":"2024-05-27T07:00:53.650368+0000","last_became_active":"2024-05-27T06:55:05.298650+0000","last_became_peered":"2024-05-27T06:55:05.298650+0000","last_unstale":"2024-05-27T07:00:53.650368+0000","last_undegraded":"2024-05-27T07:00:53.650368+0000","last_fullsized":"2024-05-27T07:00:53.650368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:30:54.161879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203525+0000","last_change":"2024-05-27T06:55:10.364006+0000","last_active":"2024-05-27T07:00:54.203525+0000","last_peered":"2024-05-27T07:00:54.203525+0000","last_clean":"2024-05-27T07:00:54.203525+0000","last_became_active":"2024-05-27T06:55:10.363620+0000","last_became_peered":"2024-05-27T06:55:10.363620+0000","last_unstale":"2024-05-27T07:00:54.203525+0000","last_undegraded":"2024-05-27T07:00:54.203525+0000","last_fullsized":"2024-05-27T07:00:54.203525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:10:18.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803276+0000","last_change":"2024-05-27T06:55:26.227377+0000","last_active":"2024-05-27T07:00:53.803276+0000","last_peered":"2024-05-27T07:00:53.803276+0000","last_clean":"2024-05-27T07:00:53.803276+0000","last_became_active":"2024-05-27T06:55:26.226869+0000","last_became_peered":"2024-05-27T06:55:26.226869+0000","last_unstale":"2024-05-27T07:00:53.803276+0000","last_undegraded":"2024-05-27T07:00:53.803276+0000","last_fullsized":"2024-05-27T07:00:53.803276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:10:40.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816169+0000","last_change":"2024-05-27T06:55:05.300986+0000","last_active":"2024-05-27T07:00:53.816169+0000","last_peered":"2024-05-27T07:00:53.816169+0000","last_clean":"2024-05-27T07:00:53.816169+0000","last_became_active":"2024-05-27T06:55:05.299946+0000","last_became_peered":"2024-05-27T06:55:05.299946+0000","last_unstale":"2024-05-27T07:00:53.816169+0000","last_undegraded":"2024-05-27T07:00:53.816169+0000","last_fullsized":"2024-05-27T07:00:53.816169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:36.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200447+0000","last_change":"2024-05-27T06:55:10.362696+0000","last_active":"2024-05-27T07:00:54.200447+0000","last_peered":"2024-05-27T07:00:54.200447+0000","last_clean":"2024-05-27T07:00:54.200447+0000","last_became_active":"2024-05-27T06:55:10.361653+0000","last_became_peered":"2024-05-27T06:55:10.361653+0000","last_unstale":"2024-05-27T07:00:54.200447+0000","last_undegraded":"2024-05-27T07:00:54.200447+0000","last_fullsized":"2024-05-27T07:00:54.200447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:38.645983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648912+0000","last_change":"2024-05-27T06:55:26.216824+0000","last_active":"2024-05-27T07:00:53.648912+0000","last_peered":"2024-05-27T07:00:53.648912+0000","last_clean":"2024-05-27T07:00:53.648912+0000","last_became_active":"2024-05-27T06:55:26.216309+0000","last_became_peered":"2024-05-27T06:55:26.216309+0000","last_unstale":"2024-05-27T07:00:53.648912+0000","last_undegraded":"2024-05-27T07:00:53.648912+0000","last_fullsized":"2024-05-27T07:00:53.648912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:56.502897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266661+0000","last_change":"2024-05-27T06:55:05.301710+0000","last_active":"2024-05-27T07:00:58.266661+0000","last_peered":"2024-05-27T07:00:58.266661+0000","last_clean":"2024-05-27T07:00:58.266661+0000","last_became_active":"2024-05-27T06:55:05.301188+0000","last_became_peered":"2024-05-27T06:55:05.301188+0000","last_unstale":"2024-05-27T07:00:58.266661+0000","last_undegraded":"2024-05-27T07:00:58.266661+0000","last_fullsized":"2024-05-27T07:00:58.266661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:02:24.790249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":8,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200387+0000","last_change":"2024-05-27T06:55:10.371493+0000","last_active":"2024-05-27T07:00:54.200387+0000","last_peered":"2024-05-27T07:00:54.200387+0000","last_clean":"2024-05-27T07:00:54.200387+0000","last_became_active":"2024-05-27T06:55:10.371048+0000","last_became_peered":"2024-05-27T06:55:10.371048+0000","last_unstale":"2024-05-27T07:00:54.200387+0000","last_undegraded":"2024-05-27T07:00:54.200387+0000","last_fullsized":"2024-05-27T07:00:54.200387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:21.452795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204558+0000","last_change":"2024-05-27T06:55:26.226751+0000","last_active":"2024-05-27T07:00:54.204558+0000","last_peered":"2024-05-27T07:00:54.204558+0000","last_clean":"2024-05-27T07:00:54.204558+0000","last_became_active":"2024-05-27T06:55:26.226215+0000","last_became_peered":"2024-05-27T06:55:26.226215+0000","last_unstale":"2024-05-27T07:00:54.204558+0000","last_undegraded":"2024-05-27T07:00:54.204558+0000","last_fullsized":"2024-05-27T07:00:54.204558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:22.366805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789874+0000","last_change":"2024-05-27T06:55:05.308772+0000","last_active":"2024-05-27T07:00:53.789874+0000","last_peered":"2024-05-27T07:00:53.789874+0000","last_clean":"2024-05-27T07:00:53.789874+0000","last_became_active":"2024-05-27T06:55:05.308078+0000","last_became_peered":"2024-05-27T06:55:05.308078+0000","last_unstale":"2024-05-27T07:00:53.789874+0000","last_undegraded":"2024-05-27T07:00:53.789874+0000","last_fullsized":"2024-05-27T07:00:53.789874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:52:29.245320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815836+0000","last_change":"2024-05-27T06:55:10.374037+0000","last_active":"2024-05-27T07:00:53.815836+0000","last_peered":"2024-05-27T07:00:53.815836+0000","last_clean":"2024-05-27T07:00:53.815836+0000","last_became_active":"2024-05-27T06:55:10.372620+0000","last_became_peered":"2024-05-27T06:55:10.372620+0000","last_unstale":"2024-05-27T07:00:53.815836+0000","last_undegraded":"2024-05-27T07:00:53.815836+0000","last_fullsized":"2024-05-27T07:00:53.815836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:11.722988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815656+0000","last_change":"2024-05-27T06:55:26.236159+0000","last_active":"2024-05-27T07:00:53.815656+0000","last_peered":"2024-05-27T07:00:53.815656+0000","last_clean":"2024-05-27T07:00:53.815656+0000","last_became_active":"2024-05-27T06:55:26.235555+0000","last_became_peered":"2024-05-27T06:55:26.235555+0000","last_unstale":"2024-05-27T07:00:53.815656+0000","last_undegraded":"2024-05-27T07:00:53.815656+0000","last_fullsized":"2024-05-27T07:00:53.815656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:39:23.561044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.792118+0000","last_change":"2024-05-27T06:55:05.300284+0000","last_active":"2024-05-27T07:00:53.792118+0000","last_peered":"2024-05-27T07:00:53.792118+0000","last_clean":"2024-05-27T07:00:53.792118+0000","last_became_active":"2024-05-27T06:55:05.299832+0000","last_became_peered":"2024-05-27T06:55:05.299832+0000","last_unstale":"2024-05-27T07:00:53.792118+0000","last_undegraded":"2024-05-27T07:00:53.792118+0000","last_fullsized":"2024-05-27T07:00:53.792118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:52.896572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803956+0000","last_change":"2024-05-27T06:55:10.376532+0000","last_active":"2024-05-27T07:00:53.803956+0000","last_peered":"2024-05-27T07:00:53.803956+0000","last_clean":"2024-05-27T07:00:53.803956+0000","last_became_active":"2024-05-27T06:55:10.373095+0000","last_became_peered":"2024-05-27T06:55:10.373095+0000","last_unstale":"2024-05-27T07:00:53.803956+0000","last_undegraded":"2024-05-27T07:00:53.803956+0000","last_fullsized":"2024-05-27T07:00:53.803956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:50:27.024281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790756+0000","last_change":"2024-05-27T06:55:26.540014+0000","last_active":"2024-05-27T07:00:53.790756+0000","last_peered":"2024-05-27T07:00:53.790756+0000","last_clean":"2024-05-27T07:00:53.790756+0000","last_became_active":"2024-05-27T06:55:26.538530+0000","last_became_peered":"2024-05-27T06:55:26.538530+0000","last_unstale":"2024-05-27T07:00:53.790756+0000","last_undegraded":"2024-05-27T07:00:53.790756+0000","last_fullsized":"2024-05-27T07:00:53.790756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:04.556414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.648046+0000","last_change":"2024-05-27T06:55:05.310791+0000","last_active":"2024-05-27T07:00:53.648046+0000","last_peered":"2024-05-27T07:00:53.648046+0000","last_clean":"2024-05-27T07:00:53.648046+0000","last_became_active":"2024-05-27T06:55:05.309636+0000","last_became_peered":"2024-05-27T06:55:05.309636+0000","last_unstale":"2024-05-27T07:00:53.648046+0000","last_undegraded":"2024-05-27T07:00:53.648046+0000","last_fullsized":"2024-05-27T07:00:53.648046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:44.912707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776506+0000","last_change":"2024-05-27T06:55:10.367656+0000","last_active":"2024-05-27T07:00:53.776506+0000","last_peered":"2024-05-27T07:00:53.776506+0000","last_clean":"2024-05-27T07:00:53.776506+0000","last_became_active":"2024-05-27T06:55:10.365625+0000","last_became_peered":"2024-05-27T06:55:10.365625+0000","last_unstale":"2024-05-27T07:00:53.776506+0000","last_undegraded":"2024-05-27T07:00:53.776506+0000","last_fullsized":"2024-05-27T07:00:53.776506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:53.235866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203958+0000","last_change":"2024-05-27T06:55:26.228777+0000","last_active":"2024-05-27T07:00:54.203958+0000","last_peered":"2024-05-27T07:00:54.203958+0000","last_clean":"2024-05-27T07:00:54.203958+0000","last_became_active":"2024-05-27T06:55:26.225616+0000","last_became_peered":"2024-05-27T06:55:26.225616+0000","last_unstale":"2024-05-27T07:00:54.203958+0000","last_undegraded":"2024-05-27T07:00:54.203958+0000","last_fullsized":"2024-05-27T07:00:54.203958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:57.142140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'9","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.214416+0000","last_change":"2024-05-27T06:55:05.295995+0000","last_active":"2024-05-27T07:00:58.214416+0000","last_peered":"2024-05-27T07:00:58.214416+0000","last_clean":"2024-05-27T07:00:58.214416+0000","last_became_active":"2024-05-27T06:55:05.294887+0000","last_became_peered":"2024-05-27T06:55:05.294887+0000","last_unstale":"2024-05-27T07:00:58.214416+0000","last_undegraded":"2024-05-27T07:00:58.214416+0000","last_fullsized":"2024-05-27T07:00:58.214416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:48.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.776481+0000","last_change":"2024-05-27T06:55:10.371524+0000","last_active":"2024-05-27T07:00:53.776481+0000","last_peered":"2024-05-27T07:00:53.776481+0000","last_clean":"2024-05-27T07:00:53.776481+0000","last_became_active":"2024-05-27T06:55:10.371089+0000","last_became_peered":"2024-05-27T06:55:10.371089+0000","last_unstale":"2024-05-27T07:00:53.776481+0000","last_undegraded":"2024-05-27T07:00:53.776481+0000","last_fullsized":"2024-05-27T07:00:53.776481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:28.274604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204687+0000","last_change":"2024-05-27T06:55:26.537969+0000","last_active":"2024-05-27T07:00:54.204687+0000","last_peered":"2024-05-27T07:00:54.204687+0000","last_clean":"2024-05-27T07:00:54.204687+0000","last_became_active":"2024-05-27T06:55:26.537305+0000","last_became_peered":"2024-05-27T06:55:26.537305+0000","last_unstale":"2024-05-27T07:00:54.204687+0000","last_undegraded":"2024-05-27T07:00:54.204687+0000","last_fullsized":"2024-05-27T07:00:54.204687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:30.798406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815963+0000","last_change":"2024-05-27T06:55:10.372760+0000","last_active":"2024-05-27T07:00:53.815963+0000","last_peered":"2024-05-27T07:00:53.815963+0000","last_clean":"2024-05-27T07:00:53.815963+0000","last_became_active":"2024-05-27T06:55:10.372212+0000","last_became_peered":"2024-05-27T06:55:10.372212+0000","last_unstale":"2024-05-27T07:00:53.815963+0000","last_undegraded":"2024-05-27T07:00:53.815963+0000","last_fullsized":"2024-05-27T07:00:53.815963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:59:17.186262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":311,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.264972+0000","last_change":"2024-05-27T06:55:05.309687+0000","last_active":"2024-05-27T07:00:58.264972+0000","last_peered":"2024-05-27T07:00:58.264972+0000","last_clean":"2024-05-27T07:00:58.264972+0000","last_became_active":"2024-05-27T06:55:05.308134+0000","last_became_peered":"2024-05-27T06:55:05.308134+0000","last_unstale":"2024-05-27T07:00:58.264972+0000","last_undegraded":"2024-05-27T07:00:58.264972+0000","last_fullsized":"2024-05-27T07:00:58.264972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:39.875004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200533+0000","last_change":"2024-05-27T06:55:26.231545+0000","last_active":"2024-05-27T07:00:54.200533+0000","last_peered":"2024-05-27T07:00:54.200533+0000","last_clean":"2024-05-27T07:00:54.200533+0000","last_became_active":"2024-05-27T06:55:26.230980+0000","last_became_peered":"2024-05-27T06:55:26.230980+0000","last_unstale":"2024-05-27T07:00:54.200533+0000","last_undegraded":"2024-05-27T07:00:54.200533+0000","last_fullsized":"2024-05-27T07:00:54.200533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:25:56.832541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801444+0000","last_change":"2024-05-27T06:55:10.372112+0000","last_active":"2024-05-27T07:00:53.801444+0000","last_peered":"2024-05-27T07:00:53.801444+0000","last_clean":"2024-05-27T07:00:53.801444+0000","last_became_active":"2024-05-27T06:55:10.368582+0000","last_became_peered":"2024-05-27T06:55:10.368582+0000","last_unstale":"2024-05-27T07:00:53.801444+0000","last_undegraded":"2024-05-27T07:00:53.801444+0000","last_fullsized":"2024-05-27T07:00:53.801444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:06.093623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.782485+0000","last_change":"2024-05-27T06:55:05.295190+0000","last_active":"2024-05-27T07:00:53.782485+0000","last_peered":"2024-05-27T07:00:53.782485+0000","last_clean":"2024-05-27T07:00:53.782485+0000","last_became_active":"2024-05-27T06:55:05.294593+0000","last_became_peered":"2024-05-27T06:55:05.294593+0000","last_unstale":"2024-05-27T07:00:53.782485+0000","last_undegraded":"2024-05-27T07:00:53.782485+0000","last_fullsized":"2024-05-27T07:00:53.782485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:22:31.937157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:55.842641+0000","last_change":"2024-05-27T06:53:04.995455+0000","last_active":"2024-05-27T07:00:55.842641+0000","last_peered":"2024-05-27T07:00:55.842641+0000","last_clean":"2024-05-27T07:00:55.842641+0000","last_became_active":"2024-05-27T06:53:04.978744+0000","last_became_peered":"2024-05-27T06:53:04.978744+0000","last_unstale":"2024-05-27T07:00:55.842641+0000","last_undegraded":"2024-05-27T07:00:55.842641+0000","last_fullsized":"2024-05-27T07:00:55.842641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:47:59.042925+0000","last_clean_scrub_stamp":"2024-05-27T06:47:59.042925+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:06.079378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.816609+0000","last_change":"2024-05-27T06:55:26.544699+0000","last_active":"2024-05-27T07:00:53.816609+0000","last_peered":"2024-05-27T07:00:53.816609+0000","last_clean":"2024-05-27T07:00:53.816609+0000","last_became_active":"2024-05-27T06:55:26.543219+0000","last_became_peered":"2024-05-27T06:55:26.543219+0000","last_unstale":"2024-05-27T07:00:53.816609+0000","last_undegraded":"2024-05-27T07:00:53.816609+0000","last_fullsized":"2024-05-27T07:00:53.816609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:59.972504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203145+0000","last_change":"2024-05-27T06:55:10.366786+0000","last_active":"2024-05-27T07:00:54.203145+0000","last_peered":"2024-05-27T07:00:54.203145+0000","last_clean":"2024-05-27T07:00:54.203145+0000","last_became_active":"2024-05-27T06:55:10.362687+0000","last_became_peered":"2024-05-27T06:55:10.362687+0000","last_unstale":"2024-05-27T07:00:54.203145+0000","last_undegraded":"2024-05-27T07:00:54.203145+0000","last_fullsized":"2024-05-27T07:00:54.203145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:44.109731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200185+0000","last_change":"2024-05-27T06:55:05.300803+0000","last_active":"2024-05-27T07:00:54.200185+0000","last_peered":"2024-05-27T07:00:54.200185+0000","last_clean":"2024-05-27T07:00:54.200185+0000","last_became_active":"2024-05-27T06:55:05.299814+0000","last_became_peered":"2024-05-27T06:55:05.299814+0000","last_unstale":"2024-05-27T07:00:54.200185+0000","last_undegraded":"2024-05-27T07:00:54.200185+0000","last_fullsized":"2024-05-27T07:00:54.200185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:43.455060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803223+0000","last_change":"2024-05-27T06:55:26.234737+0000","last_active":"2024-05-27T07:00:53.803223+0000","last_peered":"2024-05-27T07:00:53.803223+0000","last_clean":"2024-05-27T07:00:53.803223+0000","last_became_active":"2024-05-27T06:55:26.232884+0000","last_became_peered":"2024-05-27T06:55:26.232884+0000","last_unstale":"2024-05-27T07:00:53.803223+0000","last_undegraded":"2024-05-27T07:00:53.803223+0000","last_fullsized":"2024-05-27T07:00:53.803223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:33.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.791810+0000","last_change":"2024-05-27T06:55:05.310462+0000","last_active":"2024-05-27T07:00:53.791810+0000","last_peered":"2024-05-27T07:00:53.791810+0000","last_clean":"2024-05-27T07:00:53.791810+0000","last_became_active":"2024-05-27T06:55:05.309592+0000","last_became_peered":"2024-05-27T06:55:05.309592+0000","last_unstale":"2024-05-27T07:00:53.791810+0000","last_undegraded":"2024-05-27T07:00:53.791810+0000","last_fullsized":"2024-05-27T07:00:53.791810+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:40.635517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204155+0000","last_change":"2024-05-27T06:55:10.351197+0000","last_active":"2024-05-27T07:00:54.204155+0000","last_peered":"2024-05-27T07:00:54.204155+0000","last_clean":"2024-05-27T07:00:54.204155+0000","last_became_active":"2024-05-27T06:55:10.350835+0000","last_became_peered":"2024-05-27T06:55:10.350835+0000","last_unstale":"2024-05-27T07:00:54.204155+0000","last_undegraded":"2024-05-27T07:00:54.204155+0000","last_fullsized":"2024-05-27T07:00:54.204155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:17.141127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801014+0000","last_change":"2024-05-27T06:55:26.538106+0000","last_active":"2024-05-27T07:00:53.801014+0000","last_peered":"2024-05-27T07:00:53.801014+0000","last_clean":"2024-05-27T07:00:53.801014+0000","last_became_active":"2024-05-27T06:55:26.537569+0000","last_became_peered":"2024-05-27T06:55:26.537569+0000","last_unstale":"2024-05-27T07:00:53.801014+0000","last_undegraded":"2024-05-27T07:00:53.801014+0000","last_fullsized":"2024-05-27T07:00:53.801014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:01.335171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'53","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.053173+0000","last_change":"2024-05-27T06:55:39.339534+0000","last_active":"2024-05-27T07:00:54.053173+0000","last_peered":"2024-05-27T07:00:54.053173+0000","last_clean":"2024-05-27T07:00:54.053173+0000","last_became_active":"2024-05-27T06:55:39.329395+0000","last_became_peered":"2024-05-27T06:55:39.329395+0000","last_unstale":"2024-05-27T07:00:54.053173+0000","last_undegraded":"2024-05-27T07:00:54.053173+0000","last_fullsized":"2024-05-27T07:00:54.053173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:46.283281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":94,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802653+0000","last_change":"2024-05-27T06:55:10.372080+0000","last_active":"2024-05-27T07:00:53.802653+0000","last_peered":"2024-05-27T07:00:53.802653+0000","last_clean":"2024-05-27T07:00:53.802653+0000","last_became_active":"2024-05-27T06:55:10.369027+0000","last_became_peered":"2024-05-27T06:55:10.369027+0000","last_unstale":"2024-05-27T07:00:53.802653+0000","last_undegraded":"2024-05-27T07:00:53.802653+0000","last_fullsized":"2024-05-27T07:00:53.802653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:54:31.047520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649305+0000","last_change":"2024-05-27T06:55:26.224120+0000","last_active":"2024-05-27T07:00:53.649305+0000","last_peered":"2024-05-27T07:00:53.649305+0000","last_clean":"2024-05-27T07:00:53.649305+0000","last_became_active":"2024-05-27T06:55:26.223504+0000","last_became_peered":"2024-05-27T06:55:26.223504+0000","last_unstale":"2024-05-27T07:00:53.649305+0000","last_undegraded":"2024-05-27T07:00:53.649305+0000","last_fullsized":"2024-05-27T07:00:53.649305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:12.474528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783726+0000","last_change":"2024-05-27T06:55:05.309197+0000","last_active":"2024-05-27T07:00:53.783726+0000","last_peered":"2024-05-27T07:00:53.783726+0000","last_clean":"2024-05-27T07:00:53.783726+0000","last_became_active":"2024-05-27T06:55:05.308830+0000","last_became_peered":"2024-05-27T06:55:05.308830+0000","last_unstale":"2024-05-27T07:00:53.783726+0000","last_undegraded":"2024-05-27T07:00:53.783726+0000","last_fullsized":"2024-05-27T07:00:53.783726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:52.141284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844741+0000","last_change":"2024-05-27T06:55:10.359569+0000","last_active":"2024-05-27T07:00:53.844741+0000","last_peered":"2024-05-27T07:00:53.844741+0000","last_clean":"2024-05-27T07:00:53.844741+0000","last_became_active":"2024-05-27T06:55:10.359241+0000","last_became_peered":"2024-05-27T06:55:10.359241+0000","last_unstale":"2024-05-27T07:00:53.844741+0000","last_undegraded":"2024-05-27T07:00:53.844741+0000","last_fullsized":"2024-05-27T07:00:53.844741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:05.698230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783683+0000","last_change":"2024-05-27T06:55:26.544199+0000","last_active":"2024-05-27T07:00:53.783683+0000","last_peered":"2024-05-27T07:00:53.783683+0000","last_clean":"2024-05-27T07:00:53.783683+0000","last_became_active":"2024-05-27T06:55:26.542715+0000","last_became_peered":"2024-05-27T06:55:26.542715+0000","last_unstale":"2024-05-27T07:00:53.783683+0000","last_undegraded":"2024-05-27T07:00:53.783683+0000","last_fullsized":"2024-05-27T07:00:53.783683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:53.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649444+0000","last_change":"2024-05-27T06:55:39.335167+0000","last_active":"2024-05-27T07:00:53.649444+0000","last_peered":"2024-05-27T07:00:53.649444+0000","last_clean":"2024-05-27T07:00:53.649444+0000","last_became_active":"2024-05-27T06:55:39.334353+0000","last_became_peered":"2024-05-27T06:55:39.334353+0000","last_unstale":"2024-05-27T07:00:53.649444+0000","last_undegraded":"2024-05-27T07:00:53.649444+0000","last_fullsized":"2024-05-27T07:00:53.649444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:17.636407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.789897+0000","last_change":"2024-05-27T06:55:10.369899+0000","last_active":"2024-05-27T07:00:53.789897+0000","last_peered":"2024-05-27T07:00:53.789897+0000","last_clean":"2024-05-27T07:00:53.789897+0000","last_became_active":"2024-05-27T06:55:10.367261+0000","last_became_peered":"2024-05-27T06:55:10.367261+0000","last_unstale":"2024-05-27T07:00:53.789897+0000","last_undegraded":"2024-05-27T07:00:53.789897+0000","last_fullsized":"2024-05-27T07:00:53.789897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:08.997969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649098+0000","last_change":"2024-05-27T06:55:26.230103+0000","last_active":"2024-05-27T07:00:53.649098+0000","last_peered":"2024-05-27T07:00:53.649098+0000","last_clean":"2024-05-27T07:00:53.649098+0000","last_became_active":"2024-05-27T06:55:26.229598+0000","last_became_peered":"2024-05-27T06:55:26.229598+0000","last_unstale":"2024-05-27T07:00:53.649098+0000","last_undegraded":"2024-05-27T07:00:53.649098+0000","last_fullsized":"2024-05-27T07:00:53.649098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:33:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801357+0000","last_change":"2024-05-27T06:55:05.297343+0000","last_active":"2024-05-27T07:00:53.801357+0000","last_peered":"2024-05-27T07:00:53.801357+0000","last_clean":"2024-05-27T07:00:53.801357+0000","last_became_active":"2024-05-27T06:55:05.296910+0000","last_became_peered":"2024-05-27T06:55:05.296910+0000","last_unstale":"2024-05-27T07:00:53.801357+0000","last_undegraded":"2024-05-27T07:00:53.801357+0000","last_fullsized":"2024-05-27T07:00:53.801357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.434804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803270+0000","last_change":"2024-05-27T06:55:10.369215+0000","last_active":"2024-05-27T07:00:53.803270+0000","last_peered":"2024-05-27T07:00:53.803270+0000","last_clean":"2024-05-27T07:00:53.803270+0000","last_became_active":"2024-05-27T06:55:10.368842+0000","last_became_peered":"2024-05-27T06:55:10.368842+0000","last_unstale":"2024-05-27T07:00:53.803270+0000","last_undegraded":"2024-05-27T07:00:53.803270+0000","last_fullsized":"2024-05-27T07:00:53.803270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:35.889752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783228+0000","last_change":"2024-05-27T06:55:26.223787+0000","last_active":"2024-05-27T07:00:53.783228+0000","last_peered":"2024-05-27T07:00:53.783228+0000","last_clean":"2024-05-27T07:00:53.783228+0000","last_became_active":"2024-05-27T06:55:26.222936+0000","last_became_peered":"2024-05-27T06:55:26.222936+0000","last_unstale":"2024-05-27T07:00:53.783228+0000","last_undegraded":"2024-05-27T07:00:53.783228+0000","last_fullsized":"2024-05-27T07:00:53.783228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:42.299344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649930+0000","last_change":"2024-05-27T06:55:05.297874+0000","last_active":"2024-05-27T07:00:53.649930+0000","last_peered":"2024-05-27T07:00:53.649930+0000","last_clean":"2024-05-27T07:00:53.649930+0000","last_became_active":"2024-05-27T06:55:05.295082+0000","last_became_peered":"2024-05-27T06:55:05.295082+0000","last_unstale":"2024-05-27T07:00:53.649930+0000","last_undegraded":"2024-05-27T07:00:53.649930+0000","last_fullsized":"2024-05-27T07:00:53.649930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:22.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801666+0000","last_change":"2024-05-27T06:55:10.376747+0000","last_active":"2024-05-27T07:00:53.801666+0000","last_peered":"2024-05-27T07:00:53.801666+0000","last_clean":"2024-05-27T07:00:53.801666+0000","last_became_active":"2024-05-27T06:55:10.374622+0000","last_became_peered":"2024-05-27T06:55:10.374622+0000","last_unstale":"2024-05-27T07:00:53.801666+0000","last_undegraded":"2024-05-27T07:00:53.801666+0000","last_fullsized":"2024-05-27T07:00:53.801666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:51.091926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.783466+0000","last_change":"2024-05-27T06:55:26.540609+0000","last_active":"2024-05-27T07:00:53.783466+0000","last_peered":"2024-05-27T07:00:53.783466+0000","last_clean":"2024-05-27T07:00:53.783466+0000","last_became_active":"2024-05-27T06:55:26.538794+0000","last_became_peered":"2024-05-27T06:55:26.538794+0000","last_unstale":"2024-05-27T07:00:53.783466+0000","last_undegraded":"2024-05-27T07:00:53.783466+0000","last_fullsized":"2024-05-27T07:00:53.783466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:42:14.216633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.790516+0000","last_change":"2024-05-27T06:55:05.303629+0000","last_active":"2024-05-27T07:00:53.790516+0000","last_peered":"2024-05-27T07:00:53.790516+0000","last_clean":"2024-05-27T07:00:53.790516+0000","last_became_active":"2024-05-27T06:55:05.303086+0000","last_became_peered":"2024-05-27T06:55:05.303086+0000","last_unstale":"2024-05-27T07:00:53.790516+0000","last_undegraded":"2024-05-27T07:00:53.790516+0000","last_fullsized":"2024-05-27T07:00:53.790516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:26:47.361258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200234+0000","last_change":"2024-05-27T06:55:10.367061+0000","last_active":"2024-05-27T07:00:54.200234+0000","last_peered":"2024-05-27T07:00:54.200234+0000","last_clean":"2024-05-27T07:00:54.200234+0000","last_became_active":"2024-05-27T06:55:10.366666+0000","last_became_peered":"2024-05-27T06:55:10.366666+0000","last_unstale":"2024-05-27T07:00:54.200234+0000","last_undegraded":"2024-05-27T07:00:54.200234+0000","last_fullsized":"2024-05-27T07:00:54.200234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:18.512154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649466+0000","last_change":"2024-05-27T06:55:26.233399+0000","last_active":"2024-05-27T07:00:53.649466+0000","last_peered":"2024-05-27T07:00:53.649466+0000","last_clean":"2024-05-27T07:00:53.649466+0000","last_became_active":"2024-05-27T06:55:26.232796+0000","last_became_peered":"2024-05-27T06:55:26.232796+0000","last_unstale":"2024-05-27T07:00:53.649466+0000","last_undegraded":"2024-05-27T07:00:53.649466+0000","last_fullsized":"2024-05-27T07:00:53.649466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:44.827842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:58.266640+0000","last_change":"2024-05-27T06:55:05.309485+0000","last_active":"2024-05-27T07:00:58.266640+0000","last_peered":"2024-05-27T07:00:58.266640+0000","last_clean":"2024-05-27T07:00:58.266640+0000","last_became_active":"2024-05-27T06:55:05.308860+0000","last_became_peered":"2024-05-27T06:55:05.308860+0000","last_unstale":"2024-05-27T07:00:58.266640+0000","last_undegraded":"2024-05-27T07:00:58.266640+0000","last_fullsized":"2024-05-27T07:00:58.266640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:08.221700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":7,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200900+0000","last_change":"2024-05-27T06:55:10.375429+0000","last_active":"2024-05-27T07:00:54.200900+0000","last_peered":"2024-05-27T07:00:54.200900+0000","last_clean":"2024-05-27T07:00:54.200900+0000","last_became_active":"2024-05-27T06:55:10.371333+0000","last_became_peered":"2024-05-27T06:55:10.371333+0000","last_unstale":"2024-05-27T07:00:54.200900+0000","last_undegraded":"2024-05-27T07:00:54.200900+0000","last_fullsized":"2024-05-27T07:00:54.200900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:58:57.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844028+0000","last_change":"2024-05-27T06:55:26.228412+0000","last_active":"2024-05-27T07:00:53.844028+0000","last_peered":"2024-05-27T07:00:53.844028+0000","last_clean":"2024-05-27T07:00:53.844028+0000","last_became_active":"2024-05-27T06:55:26.227892+0000","last_became_peered":"2024-05-27T06:55:26.227892+0000","last_unstale":"2024-05-27T07:00:53.844028+0000","last_undegraded":"2024-05-27T07:00:53.844028+0000","last_fullsized":"2024-05-27T07:00:53.844028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:05.869109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.042149+0000","last_change":"2024-05-27T06:55:05.298384+0000","last_active":"2024-05-27T07:00:54.042149+0000","last_peered":"2024-05-27T07:00:54.042149+0000","last_clean":"2024-05-27T07:00:54.042149+0000","last_became_active":"2024-05-27T06:55:05.297757+0000","last_became_peered":"2024-05-27T06:55:05.297757+0000","last_unstale":"2024-05-27T07:00:54.042149+0000","last_undegraded":"2024-05-27T07:00:54.042149+0000","last_fullsized":"2024-05-27T07:00:54.042149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:16.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200622+0000","last_change":"2024-05-27T06:55:10.374925+0000","last_active":"2024-05-27T07:00:54.200622+0000","last_peered":"2024-05-27T07:00:54.200622+0000","last_clean":"2024-05-27T07:00:54.200622+0000","last_became_active":"2024-05-27T06:55:10.374502+0000","last_became_peered":"2024-05-27T06:55:10.374502+0000","last_unstale":"2024-05-27T07:00:54.200622+0000","last_undegraded":"2024-05-27T07:00:54.200622+0000","last_fullsized":"2024-05-27T07:00:54.200622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:33.631520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.815204+0000","last_change":"2024-05-27T06:55:26.233827+0000","last_active":"2024-05-27T07:00:53.815204+0000","last_peered":"2024-05-27T07:00:53.815204+0000","last_clean":"2024-05-27T07:00:53.815204+0000","last_became_active":"2024-05-27T06:55:26.231918+0000","last_became_peered":"2024-05-27T06:55:26.231918+0000","last_unstale":"2024-05-27T07:00:53.815204+0000","last_undegraded":"2024-05-27T07:00:53.815204+0000","last_fullsized":"2024-05-27T07:00:53.815204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:25.167539+0000","last_clean_scrub_stamp":"2024-05-27T06:55:25.167539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:54.125831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204413+0000","last_change":"2024-05-27T06:55:05.287467+0000","last_active":"2024-05-27T07:00:54.204413+0000","last_peered":"2024-05-27T07:00:54.204413+0000","last_clean":"2024-05-27T07:00:54.204413+0000","last_became_active":"2024-05-27T06:55:05.286924+0000","last_became_peered":"2024-05-27T06:55:05.286924+0000","last_unstale":"2024-05-27T07:00:54.204413+0000","last_undegraded":"2024-05-27T07:00:54.204413+0000","last_fullsized":"2024-05-27T07:00:54.204413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:04.238348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:04.238348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:40.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.200845+0000","last_change":"2024-05-27T06:55:10.378779+0000","last_active":"2024-05-27T07:00:54.200845+0000","last_peered":"2024-05-27T07:00:54.200845+0000","last_clean":"2024-05-27T07:00:54.200845+0000","last_became_active":"2024-05-27T06:55:10.376378+0000","last_became_peered":"2024-05-27T06:55:10.376378+0000","last_unstale":"2024-05-27T07:00:54.200845+0000","last_undegraded":"2024-05-27T07:00:54.200845+0000","last_fullsized":"2024-05-27T07:00:54.200845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:15.732573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803485+0000","last_change":"2024-05-27T06:55:10.377344+0000","last_active":"2024-05-27T07:00:53.803485+0000","last_peered":"2024-05-27T07:00:53.803485+0000","last_clean":"2024-05-27T07:00:53.803485+0000","last_became_active":"2024-05-27T06:55:10.372454+0000","last_became_peered":"2024-05-27T06:55:10.372454+0000","last_unstale":"2024-05-27T07:00:53.803485+0000","last_undegraded":"2024-05-27T07:00:53.803485+0000","last_fullsized":"2024-05-27T07:00:53.803485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:57:06.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650051+0000","last_change":"2024-05-27T06:55:10.368487+0000","last_active":"2024-05-27T07:00:53.650051+0000","last_peered":"2024-05-27T07:00:53.650051+0000","last_clean":"2024-05-27T07:00:53.650051+0000","last_became_active":"2024-05-27T06:55:10.368030+0000","last_became_peered":"2024-05-27T06:55:10.368030+0000","last_unstale":"2024-05-27T07:00:53.650051+0000","last_undegraded":"2024-05-27T07:00:53.650051+0000","last_fullsized":"2024-05-27T07:00:53.650051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:39.437088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.649601+0000","last_change":"2024-05-27T06:55:10.354536+0000","last_active":"2024-05-27T07:00:53.649601+0000","last_peered":"2024-05-27T07:00:53.649601+0000","last_clean":"2024-05-27T07:00:53.649601+0000","last_became_active":"2024-05-27T06:55:10.352355+0000","last_became_peered":"2024-05-27T06:55:10.352355+0000","last_unstale":"2024-05-27T07:00:53.649601+0000","last_undegraded":"2024-05-27T07:00:53.649601+0000","last_fullsized":"2024-05-27T07:00:53.649601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:49:59.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.844301+0000","last_change":"2024-05-27T06:55:39.330521+0000","last_active":"2024-05-27T07:00:53.844301+0000","last_peered":"2024-05-27T07:00:53.844301+0000","last_clean":"2024-05-27T07:00:53.844301+0000","last_became_active":"2024-05-27T06:55:39.329667+0000","last_became_peered":"2024-05-27T06:55:39.329667+0000","last_unstale":"2024-05-27T07:00:53.844301+0000","last_undegraded":"2024-05-27T07:00:53.844301+0000","last_fullsized":"2024-05-27T07:00:53.844301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:17:55.851389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.204340+0000","last_change":"2024-05-27T06:55:10.358221+0000","last_active":"2024-05-27T07:00:54.204340+0000","last_peered":"2024-05-27T07:00:54.204340+0000","last_clean":"2024-05-27T07:00:54.204340+0000","last_became_active":"2024-05-27T06:55:10.357802+0000","last_became_peered":"2024-05-27T06:55:10.357802+0000","last_unstale":"2024-05-27T07:00:54.204340+0000","last_undegraded":"2024-05-27T07:00:54.204340+0000","last_fullsized":"2024-05-27T07:00:54.204340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:31.534324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.804068+0000","last_change":"2024-05-27T06:55:10.377611+0000","last_active":"2024-05-27T07:00:53.804068+0000","last_peered":"2024-05-27T07:00:53.804068+0000","last_clean":"2024-05-27T07:00:53.804068+0000","last_became_active":"2024-05-27T06:55:10.371595+0000","last_became_peered":"2024-05-27T06:55:10.371595+0000","last_unstale":"2024-05-27T07:00:53.804068+0000","last_undegraded":"2024-05-27T07:00:53.804068+0000","last_fullsized":"2024-05-27T07:00:53.804068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:49:57.930368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.651709+0000","last_change":"2024-05-27T06:55:10.368092+0000","last_active":"2024-05-27T07:00:53.651709+0000","last_peered":"2024-05-27T07:00:53.651709+0000","last_clean":"2024-05-27T07:00:53.651709+0000","last_became_active":"2024-05-27T06:55:10.365674+0000","last_became_peered":"2024-05-27T06:55:10.365674+0000","last_unstale":"2024-05-27T07:00:53.651709+0000","last_undegraded":"2024-05-27T07:00:53.651709+0000","last_fullsized":"2024-05-27T07:00:53.651709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:38.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.802724+0000","last_change":"2024-05-27T06:55:10.359979+0000","last_active":"2024-05-27T07:00:53.802724+0000","last_peered":"2024-05-27T07:00:53.802724+0000","last_clean":"2024-05-27T07:00:53.802724+0000","last_became_active":"2024-05-27T06:55:10.359589+0000","last_became_peered":"2024-05-27T06:55:10.359589+0000","last_unstale":"2024-05-27T07:00:53.802724+0000","last_undegraded":"2024-05-27T07:00:53.802724+0000","last_fullsized":"2024-05-27T07:00:53.802724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:07:23.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203093+0000","last_change":"2024-05-27T06:55:39.327550+0000","last_active":"2024-05-27T07:00:54.203093+0000","last_peered":"2024-05-27T07:00:54.203093+0000","last_clean":"2024-05-27T07:00:54.203093+0000","last_became_active":"2024-05-27T06:55:39.326936+0000","last_became_peered":"2024-05-27T06:55:39.326936+0000","last_unstale":"2024-05-27T07:00:54.203093+0000","last_undegraded":"2024-05-27T07:00:54.203093+0000","last_fullsized":"2024-05-27T07:00:54.203093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:01.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.801509+0000","last_change":"2024-05-27T06:55:10.363917+0000","last_active":"2024-05-27T07:00:53.801509+0000","last_peered":"2024-05-27T07:00:53.801509+0000","last_clean":"2024-05-27T07:00:53.801509+0000","last_became_active":"2024-05-27T06:55:10.363480+0000","last_became_peered":"2024-05-27T06:55:10.363480+0000","last_unstale":"2024-05-27T07:00:53.801509+0000","last_undegraded":"2024-05-27T07:00:53.801509+0000","last_fullsized":"2024-05-27T07:00:53.801509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:15:05.184084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.650494+0000","last_change":"2024-05-27T06:55:10.378577+0000","last_active":"2024-05-27T07:00:53.650494+0000","last_peered":"2024-05-27T07:00:53.650494+0000","last_clean":"2024-05-27T07:00:53.650494+0000","last_became_active":"2024-05-27T06:55:10.378104+0000","last_became_peered":"2024-05-27T06:55:10.378104+0000","last_unstale":"2024-05-27T07:00:53.650494+0000","last_undegraded":"2024-05-27T07:00:53.650494+0000","last_fullsized":"2024-05-27T07:00:53.650494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:17.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:54.203784+0000","last_change":"2024-05-27T06:55:10.365608+0000","last_active":"2024-05-27T07:00:54.203784+0000","last_peered":"2024-05-27T07:00:54.203784+0000","last_clean":"2024-05-27T07:00:54.203784+0000","last_became_active":"2024-05-27T06:55:10.364754+0000","last_became_peered":"2024-05-27T06:55:10.364754+0000","last_unstale":"2024-05-27T07:00:54.203784+0000","last_undegraded":"2024-05-27T07:00:54.203784+0000","last_fullsized":"2024-05-27T07:00:54.203784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:29.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:00:53.803772+0000","last_change":"2024-05-27T06:55:10.366213+0000","last_active":"2024-05-27T07:00:53.803772+0000","last_peered":"2024-05-27T07:00:53.803772+0000","last_clean":"2024-05-27T07:00:53.803772+0000","last_became_active":"2024-05-27T06:55:10.365801+0000","last_became_peered":"2024-05-27T06:55:10.365801+0000","last_unstale":"2024-05-27T07:00:53.803772+0000","last_undegraded":"2024-05-27T07:00:53.803772+0000","last_fullsized":"2024-05-27T07:00:53.803772+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T06:55:09.298348+0000","last_clean_scrub_stamp":"2024-05-27T06:55:09.298348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:36.411610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612773,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2704,"num_read_kb":1481,"num_write":403,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1838319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678106,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1454080,"data_stored":925987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874337,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28876,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716276,"statfs":{"total":95995035648,"available":95965466624,"internally_reserved":0,"allocated":1904640,"data_stored":1326417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":928206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234095,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":531272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430328,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1679360,"data_stored":1131876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626559,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":527814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":604074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":529661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":530730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1050922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986318,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":607509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822791,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1255835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":396215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:01:03.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:01:03.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:03.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:03 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:03.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:03.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:03 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:03.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:03 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:04.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:04 smithi016 ceph-mon[31415]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:04 smithi016 ceph-mon[31415]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:04 smithi016 ceph-mon[31415]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 799 B/s wr, 15 op/s 2024-05-27T07:01:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:04 smithi164 ceph-mon[35327]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:04 smithi164 ceph-mon[35327]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:04 smithi164 ceph-mon[35327]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 799 B/s wr, 15 op/s 2024-05-27T07:01:04.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:04 smithi069 ceph-mon[35276]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:04 smithi069 ceph-mon[35276]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:01:04.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:04 smithi069 ceph-mon[35276]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 799 B/s wr, 15 op/s 2024-05-27T07:01:06.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:05 smithi016 ceph-mon[31415]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 653 B/s wr, 16 op/s 2024-05-27T07:01:06.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:05 smithi164 ceph-mon[35327]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 653 B/s wr, 16 op/s 2024-05-27T07:01:06.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:05 smithi069 ceph-mon[35276]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 653 B/s wr, 16 op/s 2024-05-27T07:01:06.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 -- 172.21.15.16:0/515636566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c10b2d0 msgr2=0x7f968c117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 --2- 172.21.15.16:0/515636566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c10b2d0 0x7f968c117800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684009ca0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 -- 172.21.15.16:0/515636566 shutdown_connections 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 --2- 172.21.15.16:0/515636566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c10b2d0 0x7f968c117800 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 --2- 172.21.15.16:0/515636566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c110960 0x7f968c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 --2- 172.21.15.16:0/515636566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10a980 0x7f968c10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 -- 172.21.15.16:0/515636566 >> 172.21.15.16:0/515636566 conn(0x7f968c077c70 msgr2=0x7f968c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 -- 172.21.15.16:0/515636566 shutdown_connections 2024-05-27T07:01:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.717+0000 7f9692a00700 1 -- 172.21.15.16:0/515636566 wait complete. 2024-05-27T07:01:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.718+0000 7f9692a00700 1 Processor -- start 2024-05-27T07:01:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.718+0000 7f9692a00700 1 -- start start 2024-05-27T07:01:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 0x7f968c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10b2d0 0x7f968c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f968c11a980 con 0x7f968c110960 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f968c11a800 con 0x7f968c10a980 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9692a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f968c11ab00 con 0x7f968c10b2d0 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9690a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9690a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37284/0 (socket says 172.21.15.16:37284) 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9690a00700 1 -- 172.21.15.16:0/2312324225 learned_addr learned my addr 172.21.15.16:0/2312324225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f968be00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 0x7f968c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.719+0000 7f9683e00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10b2d0 0x7f968c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 -- 172.21.15.16:0/2312324225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10b2d0 msgr2=0x7f968c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10b2d0 0x7f968c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 -- 172.21.15.16:0/2312324225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 msgr2=0x7f968c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 0x7f968c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 -- 172.21.15.16:0/2312324225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c1a6200 con 0x7f968c110960 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f968be00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 0x7f968c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9690a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f968400b5c0 tx=0x7f96840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684026070 con 0x7f968c110960 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f968400fb00 con 0x7f968c110960 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1a6490 con 0x7f968c110960 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.720+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684021b10 con 0x7f968c110960 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.721+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1a6970 con 0x7f968c110960 2024-05-27T07:01:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.722+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c0512b0 con 0x7f968c110960 2024-05-27T07:01:06.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.724+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f9684029070 con 0x7f968c110960 2024-05-27T07:01:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.727+0000 7f9689600700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 0x7f96700b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.727+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f96840e3bd0 con 0x7f968c110960 2024-05-27T07:01:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.727+0000 7f968be00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 0x7f96700b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.728+0000 7f968be00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 0x7f96700b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f968c07d0e0 tx=0x7f967c009040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:06.731+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9684013070 con 0x7f968c110960 2024-05-27T07:01:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.025+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f968c02e5e0 con 0x7f968c110960 2024-05-27T07:01:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.027+0000 7f9689600700 1 -- 172.21.15.16:0/2312324225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f96840ab000 con 0x7f968c110960 2024-05-27T07:01:07.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:07.019 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 msgr2=0x7f96700b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 0x7f96700b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f968c07d0e0 tx=0x7f967c009040 comp rx=0 tx=0).stop 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 msgr2=0x7f968c1a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f968400b5c0 tx=0x7f96840057c0 comp rx=0 tx=0).stop 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 shutdown_connections 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f96700b2b10 0x7f96700b5020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f968c110960 0x7f968c1a5b60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f968c10a980 0x7f968c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 --2- 172.21.15.16:0/2312324225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968c10b2d0 0x7f968c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.030+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 >> 172.21.15.16:0/2312324225 conn(0x7f968c077c70 msgr2=0x7f968c07b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.031+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 shutdown_connections 2024-05-27T07:01:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:07.031+0000 7f9692a00700 1 -- 172.21.15.16:0/2312324225 wait complete. 2024-05-27T07:01:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:01:07.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:07 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2312324225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:07 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2312324225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:07 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2312324225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:08.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:01:08.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:08 smithi016 ceph-mon[31415]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:08.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:08 smithi069 ceph-mon[35276]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:08.468 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:08.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:08 smithi164 ceph-mon[35327]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:09.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:09 smithi016 ceph-mon[31415]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:10.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:09 smithi164 ceph-mon[35327]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:10.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:09 smithi069 ceph-mon[35276]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-05-27T07:01:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 -- 172.21.15.16:0/2117945887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 msgr2=0x7f49801049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 --2- 172.21.15.16:0/2117945887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f49801049f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f49700099c0 tx=0x7f4970009cd0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 -- 172.21.15.16:0/2117945887 shutdown_connections 2024-05-27T07:01:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 --2- 172.21.15.16:0/2117945887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980109860 0x7f49801111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 --2- 172.21.15.16:0/2117945887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f49801049f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 --2- 172.21.15.16:0/2117945887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980108f10 0x7f4980109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.925+0000 7f4987000700 1 -- 172.21.15.16:0/2117945887 >> 172.21.15.16:0/2117945887 conn(0x7f4980100030 msgr2=0x7f4980102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.926+0000 7f4987000700 1 -- 172.21.15.16:0/2117945887 shutdown_connections 2024-05-27T07:01:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.926+0000 7f4987000700 1 -- 172.21.15.16:0/2117945887 wait complete. 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.927+0000 7f4987000700 1 Processor -- start 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.927+0000 7f4987000700 1 -- start start 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.927+0000 7f4987000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.927+0000 7f4987000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980108f10 0x7f498019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.927+0000 7f4987000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980109860 0x7f49801a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f4987000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4980114360 con 0x7f49801045e0 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f4987000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f49801141e0 con 0x7f4980109860 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f4987000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49801144e0 con 0x7f4980108f10 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37288/0 (socket says 172.21.15.16:37288) 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 -- 172.21.15.16:0/3827303806 learned_addr learned my addr 172.21.15.16:0/3827303806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f4977400700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980108f10 0x7f498019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f4985000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980109860 0x7f49801a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 -- 172.21.15.16:0/3827303806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980108f10 msgr2=0x7f498019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980108f10 0x7f498019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 -- 172.21.15.16:0/3827303806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980109860 msgr2=0x7f49801a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980109860 0x7f49801a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.928+0000 7f497fe00700 1 -- 172.21.15.16:0/3827303806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49801a1e40 con 0x7f49801045e0 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f497fe00700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4970005f50 tx=0x7f497000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970027070 con 0x7f49801045e0 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f497000fdd0 con 0x7f49801045e0 2024-05-27T07:01:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49801a20d0 con 0x7f49801045e0 2024-05-27T07:01:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49700173f0 con 0x7f49801045e0 2024-05-27T07:01:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.929+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a25b0 con 0x7f49801045e0 2024-05-27T07:01:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.930+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49800512b0 con 0x7f49801045e0 2024-05-27T07:01:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.933+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f497002a070 con 0x7f49801045e0 2024-05-27T07:01:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.935+0000 7f497d600700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 0x7f49640b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.935+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f49700e3e40 con 0x7f49801045e0 2024-05-27T07:01:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.935+0000 7f4977400700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 0x7f49640b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.936+0000 7f4977400700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 0x7f49640b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4968005950 tx=0x7f49680058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:10.938+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f497005fdf0 con 0x7f49801045e0 2024-05-27T07:01:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.179+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49801a9a50 con 0x7f49801045e0 2024-05-27T07:01:11.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.184+0000 7f497d600700 1 -- 172.21.15.16:0/3827303806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25084 (secure 0 0 0) 0x7f4970013070 con 0x7f49801045e0 2024-05-27T07:01:11.177 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:11.177 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T07:00:53.625840+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/702672487":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6800/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/454643589":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3317326035":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3906409731":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 msgr2=0x7f49640b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 0x7f49640b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4968005950 tx=0x7f49680058e0 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 msgr2=0x7f498019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4970005f50 tx=0x7f497000f460 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 shutdown_connections 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f49640b2b10 0x7f49640b5020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4980109860 0x7f49801a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49801045e0 0x7f498019cf70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 --2- 172.21.15.16:0/3827303806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4980108f10 0x7f498019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.188+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 >> 172.21.15.16:0/3827303806 conn(0x7f4980100030 msgr2=0x7f498010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:11.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.189+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 shutdown_connections 2024-05-27T07:01:11.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:11.189+0000 7f4987000700 1 -- 172.21.15.16:0/3827303806 wait complete. 2024-05-27T07:01:11.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:11 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3827303806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:01:11.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:11 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3827303806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:01:11.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:11 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3827303806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:01:12.139 DEBUG:tasks.fwd_scrub:Yielding 2024-05-27T07:01:12.139 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:01:12.160 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:01:12.160 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:01:12.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-27T07:01:12.161 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-27T07:01:12.161 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:01:12.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:01:12.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:12 smithi016 ceph-mon[31415]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 597 B/s wr, 19 op/s 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.450+0000 7fa1bd000640 1 -- 172.21.15.16:0/2565540552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 msgr2=0x7fa1b8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.450+0000 7fa1bd000640 1 --2- 172.21.15.16:0/2565540552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b8102ec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009900 tx=0x7fa1a402a410 comp rx=0 tx=0).stop 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 -- 172.21.15.16:0/2565540552 shutdown_connections 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 --2- 172.21.15.16:0/2565540552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b810d340 0x7fa1b810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 --2- 172.21.15.16:0/2565540552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b8102ec0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 --2- 172.21.15.16:0/2565540552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b81089e0 0x7fa1b8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 -- 172.21.15.16:0/2565540552 >> 172.21.15.16:0/2565540552 conn(0x7fa1b80fe210 msgr2=0x7fa1b8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 -- 172.21.15.16:0/2565540552 shutdown_connections 2024-05-27T07:01:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.451+0000 7fa1bd000640 1 -- 172.21.15.16:0/2565540552 wait complete. 2024-05-27T07:01:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.452+0000 7fa1bd000640 1 Processor -- start 2024-05-27T07:01:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.452+0000 7fa1bd000640 1 -- start start 2024-05-27T07:01:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 0x7fa1b819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 0x7fa1b819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8112aa0 con 0x7fa1b8102ab0 2024-05-27T07:01:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8112920 con 0x7fa1b810d340 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1bd000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8112c20 con 0x7fa1b81089e0 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33402/0 (socket says 172.21.15.16:33402) 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.453+0000 7fa1b7400640 1 -- 172.21.15.16:0/487183991 learned_addr learned my addr 172.21.15.16:0/487183991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b6a00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 0x7fa1b819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7e00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 0x7fa1b819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 -- 172.21.15.16:0/487183991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 msgr2=0x7fa1b819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 0x7fa1b819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 -- 172.21.15.16:0/487183991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 msgr2=0x7fa1b819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 0x7fa1b819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 -- 172.21.15.16:0/487183991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b819f410 con 0x7fa1b8102ab0 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7e00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 0x7fa1b819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b6a00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 0x7fa1b819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.454+0000 7fa1b7400640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9c0 tx=0x7fa1a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.455+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800cd80 con 0x7fa1b8102ab0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.455+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b819f700 con 0x7fa1b8102ab0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.455+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1a80054b0 con 0x7fa1b8102ab0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.455+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8010430 con 0x7fa1b8102ab0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.455+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a7080 con 0x7fa1b8102ab0 2024-05-27T07:01:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.456+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b806a230 con 0x7fa1b8102ab0 2024-05-27T07:01:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.459+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fa1a8004f20 con 0x7fa1b8102ab0 2024-05-27T07:01:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.461+0000 7fa1afe00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 0x7fa19c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.462+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa1a80e3350 con 0x7fa1b8102ab0 2024-05-27T07:01:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.462+0000 7fa1b6a00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 0x7fa19c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.462+0000 7fa1b6a00640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 0x7fa19c0b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1a402a930 tx=0x7fa1a4000bc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.465+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1a80aa750 con 0x7fa1b8102ab0 2024-05-27T07:01:12.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:12 smithi164 ceph-mon[35327]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 597 B/s wr, 19 op/s 2024-05-27T07:01:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:12 smithi069 ceph-mon[35276]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 597 B/s wr, 19 op/s 2024-05-27T07:01:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:12.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.697+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fa1b81a7250 con 0x7fa1b8102ab0 2024-05-27T07:01:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.704+0000 7fa1afe00640 1 -- 172.21.15.16:0/487183991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7fa1a80aa050 con 0x7fa1b8102ab0 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.707+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 msgr2=0x7fa19c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.707+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 0x7fa19c0b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1a402a930 tx=0x7fa1a4000bc0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 msgr2=0x7fa1b819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9c0 tx=0x7fa1a800ee80 comp rx=0 tx=0).stop 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 shutdown_connections 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa19c0b2a70 0x7fa19c0b4f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b810d340 0x7fa1b819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1b81089e0 0x7fa1b819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 --2- 172.21.15.16:0/487183991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b8102ab0 0x7fa1b819a5b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:12.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 >> 172.21.15.16:0/487183991 conn(0x7fa1b80fe210 msgr2=0x7fa1b806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:12.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 shutdown_connections 2024-05-27T07:01:12.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:12.708+0000 7fa1bd000640 1 -- 172.21.15.16:0/487183991 wait complete. 2024-05-27T07:01:12.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-27T07:01:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 -- 172.21.15.16:0/2070769765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 msgr2=0x7f6ddc076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2070769765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc076940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009900 tx=0x7f6dd002a490 comp rx=0 tx=0).stop 2024-05-27T07:01:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 -- 172.21.15.16:0/2070769765 shutdown_connections 2024-05-27T07:01:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2070769765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 0x7f6ddc1128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2070769765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 0x7f6ddc10ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2070769765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc076940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.007+0000 7f6de3a00640 1 -- 172.21.15.16:0/2070769765 >> 172.21.15.16:0/2070769765 conn(0x7f6ddc0ff340 msgr2=0x7f6ddc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.008+0000 7f6de3a00640 1 -- 172.21.15.16:0/2070769765 shutdown_connections 2024-05-27T07:01:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.008+0000 7f6de3a00640 1 -- 172.21.15.16:0/2070769765 wait complete. 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.009+0000 7f6de3a00640 1 Processor -- start 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.009+0000 7f6de3a00640 1 -- start start 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 0x7f6ddc1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 0x7f6ddc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc115f50 con 0x7f6ddc076530 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc115dd0 con 0x7f6ddc10db70 2024-05-27T07:01:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1160d0 con 0x7f6ddc1104d0 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33414/0 (socket says 172.21.15.16:33414) 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de1000640 1 -- 172.21.15.16:0/2485320157 learned_addr learned my addr 172.21.15.16:0/2485320157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6ddbe00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 0x7f6ddc1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.010+0000 7f6de1a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 0x7f6ddc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 -- 172.21.15.16:0/2485320157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 msgr2=0x7f6ddc19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 0x7f6ddc19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 -- 172.21.15.16:0/2485320157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 msgr2=0x7f6ddc1a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 0x7f6ddc1a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 -- 172.21.15.16:0/2485320157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc19cf30 con 0x7f6ddc076530 2024-05-27T07:01:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de1000640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009230 tx=0x7f6dd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd003f030 con 0x7f6ddc076530 2024-05-27T07:01:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd0003a80 con 0x7f6ddc076530 2024-05-27T07:01:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0035070 con 0x7f6ddc076530 2024-05-27T07:01:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.011+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc19d1c0 con 0x7f6ddc076530 2024-05-27T07:01:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.012+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc19d660 con 0x7f6ddc076530 2024-05-27T07:01:13.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.013+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc005290 con 0x7f6ddc076530 2024-05-27T07:01:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.015+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f6dd0042030 con 0x7f6ddc076530 2024-05-27T07:01:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.018+0000 7f6dd9600640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 0x7f6dc40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.018+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6dd0103210 con 0x7f6ddc076530 2024-05-27T07:01:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.018+0000 7f6ddbe00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 0x7f6dc40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.019+0000 7f6ddbe00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 0x7f6dc40b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc1124a0 tx=0x7f6dc8005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:13.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.023+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6dd00ca680 con 0x7f6ddc076530 2024-05-27T07:01:13.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.269+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6dbc005580 con 0x7f6ddc076530 2024-05-27T07:01:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.276+0000 7f6dd9600640 1 -- 172.21.15.16:0/2485320157 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f6dd00c9f80 con 0x7f6ddc076530 2024-05-27T07:01:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 msgr2=0x7f6dc40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 0x7f6dc40b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc1124a0 tx=0x7f6dc8005f90 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 msgr2=0x7f6ddc1a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009230 tx=0x7f6dd0004270 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 shutdown_connections 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6dc40b2a70 0x7f6dc40b4f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ddc1104d0 0x7f6ddc19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ddc10db70 0x7f6ddc1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 --2- 172.21.15.16:0/2485320157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc076530 0x7f6ddc1a2620 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 >> 172.21.15.16:0/2485320157 conn(0x7f6ddc0ff340 msgr2=0x7f6ddc110920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 shutdown_connections 2024-05-27T07:01:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.279+0000 7f6de3a00640 1 -- 172.21.15.16:0/2485320157 wait complete. 2024-05-27T07:01:13.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-27T07:01:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 -- 172.21.15.16:0/1925414151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518078720 msgr2=0x7f7518078b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 --2- 172.21.15.16:0/1925414151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518078720 0x7f7518078b30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7508009ca0 tx=0x7f750802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 -- 172.21.15.16:0/1925414151 shutdown_connections 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 --2- 172.21.15.16:0/1925414151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518114910 0x7f7518116d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 --2- 172.21.15.16:0/1925414151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 0x7f75181143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 --2- 172.21.15.16:0/1925414151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518078720 0x7f7518078b30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 -- 172.21.15.16:0/1925414151 >> 172.21.15.16:0/1925414151 conn(0x7f7518101460 msgr2=0x7f7518103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 -- 172.21.15.16:0/1925414151 shutdown_connections 2024-05-27T07:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.583+0000 7f751ca00640 1 -- 172.21.15.16:0/1925414151 wait complete. 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.584+0000 7f751ca00640 1 Processor -- start 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 -- start start 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 0x7f75181a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 0x7f75181a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f751811a1a0 con 0x7f7518114910 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f751811a020 con 0x7f7518078720 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f751ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f751811a320 con 0x7f7518111fb0 2024-05-27T07:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f7517e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f7517e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33432/0 (socket says 172.21.15.16:33432) 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.585+0000 7f7517e00640 1 -- 172.21.15.16:0/1413936287 learned_addr learned my addr 172.21.15.16:0/1413936287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517400640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 0x7f75181a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7516a00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 0x7f75181a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 -- 172.21.15.16:0/1413936287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 msgr2=0x7f75181a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 0x7f75181a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 -- 172.21.15.16:0/1413936287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 msgr2=0x7f75181a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 0x7f75181a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 -- 172.21.15.16:0/1413936287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75181a1100 con 0x7f7518114910 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517400640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 0x7f75181a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f7517e00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f751000e970 tx=0x7f751000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.586+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000ccc0 con 0x7f7518114910 2024-05-27T07:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.587+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f751000ce60 con 0x7f7518114910 2024-05-27T07:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.587+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75181a13f0 con 0x7f7518114910 2024-05-27T07:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.587+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510010430 con 0x7f7518114910 2024-05-27T07:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.587+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181ad6d0 con 0x7f7518114910 2024-05-27T07:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.588+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7518050860 con 0x7f7518114910 2024-05-27T07:01:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.592+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f7510004ef0 con 0x7f7518114910 2024-05-27T07:01:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.594+0000 7f750fe00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 0x7f74fc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.594+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f75100e3340 con 0x7f7518114910 2024-05-27T07:01:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.594+0000 7f7517400640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 0x7f74fc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.595+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7510054df0 con 0x7f7518114910 2024-05-27T07:01:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.595+0000 7f7517400640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 0x7f74fc0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7508000f80 tx=0x7f7508009430 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.815+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f75181a1f90 con 0x7f7518114910 2024-05-27T07:01:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.821+0000 7f750fe00640 1 -- 172.21.15.16:0/1413936287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f75181a1f90 con 0x7f7518114910 2024-05-27T07:01:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 msgr2=0x7f74fc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 0x7f74fc0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7508000f80 tx=0x7f7508009430 comp rx=0 tx=0).stop 2024-05-27T07:01:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 msgr2=0x7f75181a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f751000e970 tx=0x7f751000ed70 comp rx=0 tx=0).stop 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 shutdown_connections 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f74fc0b2a70 0x7f74fc0b4f80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518114910 0x7f75181a0a60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518111fb0 0x7f75181a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 --2- 172.21.15.16:0/1413936287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7518078720 0x7f75181a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 >> 172.21.15.16:0/1413936287 conn(0x7f7518101460 msgr2=0x7f751807b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.825+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 shutdown_connections 2024-05-27T07:01:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:13.826+0000 7f751ca00640 1 -- 172.21.15.16:0/1413936287 wait complete. 2024-05-27T07:01:13.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-27T07:01:14.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/487183991' entity='client.admin' 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2485320157' entity='client.admin' 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:13 smithi164 ceph-mon[35327]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:14.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/487183991' entity='client.admin' 2024-05-27T07:01:14.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2485320157' entity='client.admin' 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:13 smithi016 ceph-mon[31415]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 -- 172.21.15.16:0/1337033930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74810bce0 msgr2=0x7fd748105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 --2- 172.21.15.16:0/1337033930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74810bce0 0x7fd748105880 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009900 tx=0x7fd73c02a410 comp rx=0 tx=0).stop 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 -- 172.21.15.16:0/1337033930 shutdown_connections 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 --2- 172.21.15.16:0/1337033930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd748106430 0x7fd748112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 --2- 172.21.15.16:0/1337033930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 0x7fd748105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 --2- 172.21.15.16:0/1337033930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74810bce0 0x7fd748105880 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 -- 172.21.15.16:0/1337033930 >> 172.21.15.16:0/1337033930 conn(0x7fd748101380 msgr2=0x7fd7481037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.125+0000 7fd74f000640 1 -- 172.21.15.16:0/1337033930 shutdown_connections 2024-05-27T07:01:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.126+0000 7fd74f000640 1 -- 172.21.15.16:0/1337033930 wait complete. 2024-05-27T07:01:14.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/487183991' entity='client.admin' 2024-05-27T07:01:14.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2485320157' entity='client.admin' 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:14.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:13 smithi069 ceph-mon[35276]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.127+0000 7fd74f000640 1 Processor -- start 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.127+0000 7fd74f000640 1 -- start start 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.127+0000 7fd74f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.127+0000 7fd74f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd74810bce0 0x7fd7481a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74f000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 0x7fd74819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd748115dc0 con 0x7fd748106430 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74f000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd748115c40 con 0x7fd74810c180 2024-05-27T07:01:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd748115f40 con 0x7fd74810bce0 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33450/0 (socket says 172.21.15.16:33450) 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 -- 172.21.15.16:0/1996536106 learned_addr learned my addr 172.21.15.16:0/1996536106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74e600640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 0x7fd74819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74d200640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd74810bce0 0x7fd7481a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 -- 172.21.15.16:0/1996536106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd74810bce0 msgr2=0x7fd7481a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd74810bce0 0x7fd7481a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 -- 172.21.15.16:0/1996536106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 msgr2=0x7fd74819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 0x7fd74819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.128+0000 7fd74dc00640 1 -- 172.21.15.16:0/1996536106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74819cdd0 con 0x7fd748106430 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd74dc00640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd73c02a930 tx=0x7fd73c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c03f070 con 0x7fd748106430 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd73c003e10 con 0x7fd748106430 2024-05-27T07:01:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c035030 con 0x7fd748106430 2024-05-27T07:01:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74819d000 con 0x7fd748106430 2024-05-27T07:01:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.129+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74819d4a0 con 0x7fd748106430 2024-05-27T07:01:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.130+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748106fc0 con 0x7fd748106430 2024-05-27T07:01:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.133+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fd73c042070 con 0x7fd748106430 2024-05-27T07:01:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.135+0000 7fd746000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 0x7fd7300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.135+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd73c103410 con 0x7fd748106430 2024-05-27T07:01:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.135+0000 7fd74d200640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 0x7fd7300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.136+0000 7fd74d200640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 0x7fd7300b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd734005fd0 tx=0x7fd73400b510 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:14.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.139+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd73c0ca880 con 0x7fd748106430 2024-05-27T07:01:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.388+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd748101ea0 con 0x7fd7300b2a70 2024-05-27T07:01:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.552+0000 7fd746000640 1 -- 172.21.15.16:0/1996536106 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd748101ea0 con 0x7fd7300b2a70 2024-05-27T07:01:14.545 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-05-27T07:01:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.555+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 msgr2=0x7fd7300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.555+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 0x7fd7300b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd734005fd0 tx=0x7fd73400b510 comp rx=0 tx=0).stop 2024-05-27T07:01:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.555+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 msgr2=0x7fd7481a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.555+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd73c02a930 tx=0x7fd73c003c20 comp rx=0 tx=0).stop 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 shutdown_connections 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd7300b2a70 0x7fd7300b4f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74810c180 0x7fd74819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd74810bce0 0x7fd7481a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 --2- 172.21.15.16:0/1996536106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd748106430 0x7fd7481a2520 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 >> 172.21.15.16:0/1996536106 conn(0x7fd748101380 msgr2=0x7fd748101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 shutdown_connections 2024-05-27T07:01:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.556+0000 7fd74f000640 1 -- 172.21.15.16:0/1996536106 wait complete. 2024-05-27T07:01:14.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-27T07:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 -- 172.21.15.16:0/1473333992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 msgr2=0x7fe7901127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 --2- 172.21.15.16:0/1473333992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe7901127c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe788009ca0 tx=0x7fe78802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 -- 172.21.15.16:0/1473333992 shutdown_connections 2024-05-27T07:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 --2- 172.21.15.16:0/1473333992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe7901127c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 --2- 172.21.15.16:0/1473333992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 0x7fe790105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 --2- 172.21.15.16:0/1473333992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 0x7fe790105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 -- 172.21.15.16:0/1473333992 >> 172.21.15.16:0/1473333992 conn(0x7fe790101300 msgr2=0x7fe790103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 -- 172.21.15.16:0/1473333992 shutdown_connections 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.866+0000 7fe796c00640 1 -- 172.21.15.16:0/1473333992 wait complete. 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.867+0000 7fe796c00640 1 Processor -- start 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.867+0000 7fe796c00640 1 -- start start 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 0x7fe79019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 0x7fe7901a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe790115d90 con 0x7fe7901063f0 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe790115c10 con 0x7fe79010bca0 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe790115f10 con 0x7fe79010c140 2024-05-27T07:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33468/0 (socket says 172.21.15.16:33468) 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 -- 172.21.15.16:0/2608857344 learned_addr learned my addr 172.21.15.16:0/2608857344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe796200640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 0x7fe7901a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe794e00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 0x7fe79019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 -- 172.21.15.16:0/2608857344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 msgr2=0x7fe7901a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 0x7fe7901a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 -- 172.21.15.16:0/2608857344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 msgr2=0x7fe79019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 0x7fe79019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.868+0000 7fe795800640 1 -- 172.21.15.16:0/2608857344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7901a15d0 con 0x7fe7901063f0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe795800640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe78400d770 tx=0x7fe78400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784017070 con 0x7fe7901063f0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7901a18c0 con 0x7fe7901063f0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe784009c30 con 0x7fe7901063f0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400f620 con 0x7fe7901063f0 2024-05-27T07:01:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.869+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a9240 con 0x7fe7901063f0 2024-05-27T07:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.870+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790050860 con 0x7fe7901063f0 2024-05-27T07:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.874+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fe78401a070 con 0x7fe7901063f0 2024-05-27T07:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.875+0000 7fe78e000640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 0x7fe7780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.875+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe7840e28c0 con 0x7fe7901063f0 2024-05-27T07:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.875+0000 7fe794e00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 0x7fe7780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.876+0000 7fe794e00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 0x7fe7780b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe79010ad60 tx=0x7fe77c005c90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:14.877+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe784009de0 con 0x7fe7901063f0 2024-05-27T07:01:15.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1413936287' entity='client.admin' 2024-05-27T07:01:15.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:15.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:15.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:15.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:15.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:14 smithi016 ceph-mon[31415]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1413936287' entity='client.admin' 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:15.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:14 smithi164 ceph-mon[35327]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.118+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fe790063100 con 0x7fe7780b2a70 2024-05-27T07:01:15.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1413936287' entity='client.admin' 2024-05-27T07:01:15.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:01:15.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:01:15.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:01:15.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:01:15.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:14 smithi069 ceph-mon[35276]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 -- 172.21.15.16:0/102092537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed30108550 msgr2=0x7fed3010feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 --2- 172.21.15.16:0/102092537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed30108550 0x7fed3010feb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fed2800b0d0 tx=0x7fed2800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 -- 172.21.15.16:0/102092537 shutdown_connections 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 --2- 172.21.15.16:0/102092537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed30108550 0x7fed3010feb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 --2- 172.21.15.16:0/102092537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed301032d0 0x7fed301036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 --2- 172.21.15.16:0/102092537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30107c00 0x7fed30108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 -- 172.21.15.16:0/102092537 >> 172.21.15.16:0/102092537 conn(0x7fed30078ab0 msgr2=0x7fed30078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 -- 172.21.15.16:0/102092537 shutdown_connections 2024-05-27T07:01:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.162+0000 7fed2fe00700 1 -- 172.21.15.16:0/102092537 wait complete. 2024-05-27T07:01:15.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 Processor -- start 2024-05-27T07:01:15.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 -- start start 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed30107c00 0x7fed3019d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30108550 0x7fed301a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed30113030 con 0x7fed301032d0 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112eb0 con 0x7fed30108550 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.164+0000 7fed2fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed301131b0 con 0x7fed30107c00 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33494/0 (socket says 172.21.15.16:33494) 2024-05-27T07:01:15.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 -- 172.21.15.16:0/1198270169 learned_addr learned my addr 172.21.15.16:0/1198270169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2e000700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed30107c00 0x7fed3019d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2f400700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30108550 0x7fed301a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 -- 172.21.15.16:0/1198270169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed30107c00 msgr2=0x7fed3019d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed30107c00 0x7fed3019d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 -- 172.21.15.16:0/1198270169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30108550 msgr2=0x7fed301a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30108550 0x7fed301a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.165+0000 7fed2ea00700 1 -- 172.21.15.16:0/1198270169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed301a1bf0 con 0x7fed301032d0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.166+0000 7fed2ea00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fed2000d950 tx=0x7fed2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.166+0000 7fed27400700 1 -- 172.21.15.16:0/1198270169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20016070 con 0x7fed301032d0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.166+0000 7fed27400700 1 -- 172.21.15.16:0/1198270169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed2000fd20 con 0x7fed301032d0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.166+0000 7fed27400700 1 -- 172.21.15.16:0/1198270169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2000e3b0 con 0x7fed301032d0 2024-05-27T07:01:15.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.166+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed301a1ee0 con 0x7fed301032d0 2024-05-27T07:01:15.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.167+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301a9800 con 0x7fed301032d0 2024-05-27T07:01:15.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.171+0000 7fed27400700 1 -- 172.21.15.16:0/1198270169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fed20019070 con 0x7fed301032d0 2024-05-27T07:01:15.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.172+0000 7fed441bdb80 1 Processor -- start 2024-05-27T07:01:15.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 -- start start 2024-05-27T07:01:15.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed27400700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 0x7fed140b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e02f6daec0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e02f6caa60 con 0x55e02f7ac490 2024-05-27T07:01:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed441bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e02f6ceaa0 con 0x55e02f62dec0 2024-05-27T07:01:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.173+0000 7fed27400700 1 -- 172.21.15.16:0/1198270169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fed200e2780 con 0x7fed301032d0 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2ea00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 0x7fed140b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33498/0 (socket says 172.21.15.16:33498) 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 -- 172.21.15.16:0/551062518 learned_addr learned my addr 172.21.15.16:0/551062518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2e000700 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 -- 172.21.15.16:0/551062518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 msgr2=0x55e02f7abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 -- 172.21.15.16:0/551062518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 msgr2=0x55e02f7b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.174+0000 7fed2f400700 1 -- 172.21.15.16:0/551062518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02f6934a0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.175+0000 7fed2f400700 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed28009610 tx=0x7fed28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869aff5db97cff24 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.175+0000 7fed2ea00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 0x7fed140b5020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed200df9c0 tx=0x7fed20010f30 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.175+0000 7fed1be00700 1 -- 172.21.15.16:0/551062518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28004c40 con 0x55e02f7ac8a0 2024-05-27T07:01:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.175+0000 7fed1be00700 1 -- 172.21.15.16:0/551062518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed28004de0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.175+0000 7fed1be00700 1 -- 172.21.15.16:0/551062518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28005320 con 0x55e02f7ac8a0 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.176+0000 7fed441bdb80 1 -- 172.21.15.16:0/551062518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 msgr2=0x55e02f7b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.176+0000 7fed441bdb80 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed28009610 tx=0x7fed28009780 comp rx=0 tx=0).stop 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 -- 172.21.15.16:0/551062518 shutdown_connections 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7b90a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 --2- 172.21.15.16:0/551062518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 -- 172.21.15.16:0/551062518 >> 172.21.15.16:0/551062518 conn(0x55e02f68e040 msgr2=0x55e02f7a7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 -- 172.21.15.16:0/551062518 shutdown_connections 2024-05-27T07:01:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.177+0000 7fed441bdb80 1 -- 172.21.15.16:0/551062518 wait complete. 2024-05-27T07:01:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.178+0000 7fed441bdb80 1 -- start start 2024-05-27T07:01:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.178+0000 7fed441bdb80 1 Processor -- start 2024-05-27T07:01:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e02f5d1020 con 0x55e02f7ac8a0 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e02f66b7e0 con 0x55e02f7ac490 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e02f7b99b0 con 0x55e02f62dec0 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed441bdb80 10 client.0 _ll_register_callbacks cb 0x55e02f674cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33502/0 (socket says 172.21.15.16:33502) 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed2f400700 1 -- 172.21.15.16:0/2043944140 learned_addr learned my addr 172.21.15.16:0/2043944140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.179+0000 7fed2f400700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2e000700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 -- 172.21.15.16:0/2043944140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 msgr2=0x55e02f7c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 -- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 msgr2=0x55e02f7c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 -- 172.21.15.16:0/2043944140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02f61a950 con 0x55e02f7ac8a0 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2e000700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed2f400700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed28007df0 tx=0x7fed28007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f684e88747bdc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.180+0000 7fed18c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280077c0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fed28007960 con 0x55e02f7ac8a0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed28004de0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed280051a0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed441bdb80 10 client.15480 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.181+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e02f69b0e0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.182+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed28004de0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.182+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fed28037030 con 0x55e02f7ac8a0 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 10 client.15480 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 20 client.15480 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f7cf040 0x55e02f7d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 4 client.15480 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:01:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02f6dce10 con 0x55e02f7cf040 2024-05-27T07:01:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.183+0000 7fed2e000700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f7cf040 0x55e02f7d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.185+0000 7fed2e000700 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f7cf040 0x55e02f7d1460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59f98f70b6c5a12 server_cookie=a4dbbbfd18b9846d in_seq=0 out_seq=0 2024-05-27T07:01:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.185+0000 7fed18c00700 10 client.15480 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed18c00700 1 -- 172.21.15.16:0/2043944140 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e02f6dce10 con 0x55e02f7cf040 2024-05-27T07:01:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed18c00700 10 client.15480 handle_command_reply: tid=0 2024-05-27T07:01:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 client.15480 shutdown 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 msgr2=0x55e02f7bf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed28007df0 tx=0x7fed28007e50 comp rx=0 tx=0).stop 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 shutdown_connections 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f7cf040 0x55e02f7d1460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f7bf520 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 --2- 172.21.15.16:0/2043944140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f7c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 >> 172.21.15.16:0/2043944140 conn(0x55e02f68e040 msgr2=0x55e02f7a7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.187+0000 7fed18c00700 0 client.15480 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.188+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 shutdown_connections 2024-05-27T07:01:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.188+0000 7fed441bdb80 1 -- 172.21.15.16:0/2043944140 wait complete. 2024-05-27T07:01:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.188+0000 7fed441bdb80 20 client.15480 trim_cache size 0 max 16384 2024-05-27T07:01:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.215+0000 7fed441bdb80 1 Processor -- start 2024-05-27T07:01:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.215+0000 7fed441bdb80 1 -- start start 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac8a0 0x55e02f7cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed301ae090 con 0x55e02f7ac490 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed301afbd0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed441bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed301ae760 con 0x55e02f62dec0 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed2e000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed2e000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33512/0 (socket says 172.21.15.16:33512) 2024-05-27T07:01:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.216+0000 7fed2e000700 1 -- 172.21.15.16:0/256130957 learned_addr learned my addr 172.21.15.16:0/256130957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2f400700 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2f400700 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac8a0 0x55e02f7cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 -- 172.21.15.16:0/256130957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 msgr2=0x55e02f7bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7bc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 -- 172.21.15.16:0/256130957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac8a0 msgr2=0x55e02f7cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac8a0 0x55e02f7cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 -- 172.21.15.16:0/256130957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02f7bf940 con 0x55e02f7ac490 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.217+0000 7fed2e000700 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3328bfb023df30a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.218+0000 7fed1b400700 1 -- 172.21.15.16:0/256130957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c004030 con 0x55e02f7ac490 2024-05-27T07:01:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.218+0000 7fed1b400700 1 -- 172.21.15.16:0/256130957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed1c004830 con 0x55e02f7ac490 2024-05-27T07:01:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.218+0000 7fed1b400700 1 -- 172.21.15.16:0/256130957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c005420 con 0x55e02f7ac490 2024-05-27T07:01:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 -- 172.21.15.16:0/256130957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 msgr2=0x55e02f7bccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c00b410 comp rx=0 tx=0).stop 2024-05-27T07:01:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 -- 172.21.15.16:0/256130957 shutdown_connections 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac8a0 0x55e02f7cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f7bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 --2- 172.21.15.16:0/256130957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac490 0x55e02f7bccc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 -- 172.21.15.16:0/256130957 >> 172.21.15.16:0/256130957 conn(0x55e02f68e040 msgr2=0x55e02f7a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 -- 172.21.15.16:0/256130957 shutdown_connections 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.219+0000 7fed441bdb80 1 -- 172.21.15.16:0/256130957 wait complete. 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.220+0000 7fed441bdb80 1 -- start start 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.220+0000 7fed441bdb80 1 Processor -- start 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f82da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f82df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed301af500 con 0x55e02f7ac8a0 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed301aee30 con 0x55e02f7ac490 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed301ac030 con 0x55e02f62dec0 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed441bdb80 10 client.0 _ll_register_callbacks cb 0x55e02f674cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed2f400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33524/0 (socket says 172.21.15.16:33524) 2024-05-27T07:01:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.221+0000 7fed2f400700 1 -- 172.21.15.16:0/3242873377 learned_addr learned my addr 172.21.15.16:0/3242873377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f82da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2e000700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f82df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 -- 172.21.15.16:0/3242873377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 msgr2=0x55e02f82da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f82da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 -- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 msgr2=0x55e02f82df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f82df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 -- 172.21.15.16:0/3242873377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02f832ec0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed2f400700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fed28006070 tx=0x7fed28006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18620cb1390b2ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.222+0000 7fed18c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280076c0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fed28007860 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c004830 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed28007a60 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed441bdb80 10 client.15486 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e02f83a1d0 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.223+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed1c004830 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fed28037070 con 0x55e02f7ac8a0 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 10 client.15486 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 20 client.15486 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f83a7d0 0x55e02f83cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 4 client.15486 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.224+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e02f6d65f0 con 0x55e02f83a7d0 2024-05-27T07:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.225+0000 7fed2e000700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f83a7d0 0x55e02f83cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.226+0000 7fed2e000700 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f83a7d0 0x55e02f83cbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdb63a8716ab87e server_cookie=21db669f0eefacfd in_seq=0 out_seq=0 2024-05-27T07:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.226+0000 7fed18c00700 10 client.15486 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed18c00700 1 -- 172.21.15.16:0/3242873377 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e02f6d65f0 con 0x55e02f83a7d0 2024-05-27T07:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed18c00700 10 client.15486 handle_command_reply: tid=0 2024-05-27T07:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 client.15486 shutdown 2024-05-27T07:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 msgr2=0x55e02f832900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fed28006070 tx=0x7fed28006000 comp rx=0 tx=0).stop 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 shutdown_connections 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55e02f83a7d0 0x55e02f83cbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02f7ac8a0 0x55e02f832900 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e02f62dec0 0x55e02f82da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 --2- 172.21.15.16:0/3242873377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e02f7ac490 0x55e02f82df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 >> 172.21.15.16:0/3242873377 conn(0x55e02f68e040 msgr2=0x55e02f7ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.229+0000 7fed18c00700 0 client.15486 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.230+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 shutdown_connections 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.230+0000 7fed441bdb80 1 -- 172.21.15.16:0/3242873377 wait complete. 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.230+0000 7fed441bdb80 20 client.15486 trim_cache size 0 max 16384 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "6464d79d-55f0-4170-94d8-d95a960d4bfc", 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:01:15.223 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 msgr2=0x7fed140b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 0x7fed140b5020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed200df9c0 tx=0x7fed20010f30 comp rx=0 tx=0).stop 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 msgr2=0x7fed3019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fed2000d950 tx=0x7fed2000dc60 comp rx=0 tx=0).stop 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 shutdown_connections 2024-05-27T07:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fed140b2b10 0x7fed140b5020 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30108550 0x7fed301a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed301032d0 0x7fed3019cd20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 --2- 172.21.15.16:0/1198270169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed30107c00 0x7fed3019d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 >> 172.21.15.16:0/1198270169 conn(0x7fed30078ab0 msgr2=0x7fed3010ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 shutdown_connections 2024-05-27T07:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.231+0000 7fed2fe00700 1 -- 172.21.15.16:0/1198270169 wait complete. 2024-05-27T07:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.259+0000 7fe78e000640 1 -- 172.21.15.16:0/2608857344 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fe790063100 con 0x7fe7780b2a70 2024-05-27T07:01:15.252 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 msgr2=0x7fe7780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 0x7fe7780b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe79010ad60 tx=0x7fe77c005c90 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 msgr2=0x7fe79019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe78400d770 tx=0x7fe78400dc30 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 shutdown_connections 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe7780b2a70 0x7fe7780b4f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe79010c140 0x7fe7901a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe79010bca0 0x7fe79019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.262+0000 7fe796c00640 1 --2- 172.21.15.16:0/2608857344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7901063f0 0x7fe79019c770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.263+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 >> 172.21.15.16:0/2608857344 conn(0x7fe790101300 msgr2=0x7fe7901029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.263+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 shutdown_connections 2024-05-27T07:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.263+0000 7fe796c00640 1 -- 172.21.15.16:0/2608857344 wait complete. 2024-05-27T07:01:15.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T07:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 -- 172.21.15.16:0/1529054410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f429810beb0 msgr2=0x7f4298105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 --2- 172.21.15.16:0/1529054410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f429810beb0 0x7f4298105a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4288009900 tx=0x7f428802a400 comp rx=0 tx=0).stop 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 -- 172.21.15.16:0/1529054410 shutdown_connections 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 --2- 172.21.15.16:0/1529054410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4298106600 0x7f42981129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 --2- 172.21.15.16:0/1529054410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 0x7f4298105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 --2- 172.21.15.16:0/1529054410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f429810beb0 0x7f4298105a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 -- 172.21.15.16:0/1529054410 >> 172.21.15.16:0/1529054410 conn(0x7f42981016b0 msgr2=0x7f4298103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 -- 172.21.15.16:0/1529054410 shutdown_connections 2024-05-27T07:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.560+0000 7f429e600640 1 -- 172.21.15.16:0/1529054410 wait complete. 2024-05-27T07:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.562+0000 7f429e600640 1 Processor -- start 2024-05-27T07:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.562+0000 7f429e600640 1 -- start start 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 0x7f429819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 0x7f42981a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4298115f60 con 0x7f429810beb0 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4298115de0 con 0x7f429810c350 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f429e600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42981160e0 con 0x7f4298106600 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f4296a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f4296a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33542/0 (socket says 172.21.15.16:33542) 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f4296a00640 1 -- 172.21.15.16:0/1161621073 learned_addr learned my addr 172.21.15.16:0/1161621073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f4297400640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 0x7f429819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.563+0000 7f4297e00640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 0x7f42981a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 -- 172.21.15.16:0/1161621073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 msgr2=0x7f429819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 0x7f429819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 -- 172.21.15.16:0/1161621073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 msgr2=0x7f42981a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 0x7f42981a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 -- 172.21.15.16:0/1161621073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42981a1900 con 0x7f429810beb0 2024-05-27T07:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4297400640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 0x7f429819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f4296a00640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f428400e9f0 tx=0x7f428400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428400ccc0 con 0x7f429810beb0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42840035f0 con 0x7f429810beb0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42981a1bf0 con 0x7f429810beb0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.564+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428400ce60 con 0x7f429810beb0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.565+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42981a20f0 con 0x7f429810beb0 2024-05-27T07:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.566+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429819ddf0 con 0x7f429810beb0 2024-05-27T07:01:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.569+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f4284004000 con 0x7f429810beb0 2024-05-27T07:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.572+0000 7f428fe00640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 0x7f427c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.572+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f42840e3140 con 0x7f429810beb0 2024-05-27T07:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.572+0000 7f4297400640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 0x7f427c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.573+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42840aa540 con 0x7f429810beb0 2024-05-27T07:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.573+0000 7f4297400640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 0x7f427c0b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42880057c0 tx=0x7f4288005de0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.827+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f42981021d0 con 0x7f427c0b2a70 2024-05-27T07:01:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.846+0000 7f428fe00640 1 -- 172.21.15.16:0/1161621073 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f42981021d0 con 0x7f427c0b2a70 2024-05-27T07:01:15.838 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:01:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.848+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 msgr2=0x7f427c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.848+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 0x7f427c0b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42880057c0 tx=0x7f4288005de0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.848+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 msgr2=0x7f429819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.848+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f428400e9f0 tx=0x7f428400eeb0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 shutdown_connections 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f427c0b2a70 0x7f427c0b4f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429810c350 0x7f42981a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429810beb0 0x7f429819d000 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 --2- 172.21.15.16:0/1161621073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4298106600 0x7f429819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 >> 172.21.15.16:0/1161621073 conn(0x7f42981016b0 msgr2=0x7f4298101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:15.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 shutdown_connections 2024-05-27T07:01:15.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:15.849+0000 7f429e600640 1 -- 172.21.15.16:0/1161621073 wait complete. 2024-05-27T07:01:15.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T07:01:16.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:15 smithi164 ceph-mon[35327]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:16.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:15 smithi164 ceph-mon[35327]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T07:01:16.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:15 smithi016 ceph-mon[31415]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:16.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:15 smithi016 ceph-mon[31415]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T07:01:16.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:15 smithi069 ceph-mon[35276]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:16.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:15 smithi069 ceph-mon[35276]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 -- 172.21.15.16:0/3057442141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 msgr2=0x7f3f6810fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 --2- 172.21.15.16:0/3057442141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f6810fdb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009900 tx=0x7f3f5002a480 comp rx=0 tx=0).stop 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 -- 172.21.15.16:0/3057442141 shutdown_connections 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 --2- 172.21.15.16:0/3057442141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f68112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 --2- 172.21.15.16:0/3057442141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f6810fdb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 --2- 172.21.15.16:0/3057442141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 0x7f3f68075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 -- 172.21.15.16:0/3057442141 >> 172.21.15.16:0/3057442141 conn(0x7f3f68079240 msgr2=0x7f3f68079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 -- 172.21.15.16:0/3057442141 shutdown_connections 2024-05-27T07:01:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.157+0000 7f3f67200640 1 -- 172.21.15.16:0/3057442141 wait complete. 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.158+0000 7f3f67200640 1 Processor -- start 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 -- start start 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 0x7f3f681a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f6819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68115d80 con 0x7f3f6810d990 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68115c00 con 0x7f3f68075480 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f67200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68115f00 con 0x7f3f681102f0 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.159+0000 7f3f65400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33564/0 (socket says 172.21.15.16:33564) 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 -- 172.21.15.16:0/2746088916 learned_addr learned my addr 172.21.15.16:0/2746088916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65e00640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 0x7f3f681a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f66800640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f6819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 -- 172.21.15.16:0/2746088916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 msgr2=0x7f3f6819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f6819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 -- 172.21.15.16:0/2746088916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 msgr2=0x7f3f681a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 0x7f3f681a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f65400640 1 -- 172.21.15.16:0/2746088916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6819cca0 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.160+0000 7f3f66800640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f6819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f65400640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002a9a0 tx=0x7f3f50003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5003f070 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f50003e10 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6819cf30 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50035030 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.161+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6819d470 con 0x7f3f6810d990 2024-05-27T07:01:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.162+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68076750 con 0x7f3f6810d990 2024-05-27T07:01:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.165+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f3f50042070 con 0x7f3f6810d990 2024-05-27T07:01:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.166+0000 7f3f5ea00640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 0x7f3f4c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.166+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f3f50103380 con 0x7f3f6810d990 2024-05-27T07:01:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.167+0000 7f3f65e00640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 0x7f3f4c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.168+0000 7f3f65e00640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 0x7f3f4c0b5000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f680770d0 tx=0x7f3f58009670 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.171+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f500ca7f0 con 0x7f3f6810d990 2024-05-27T07:01:16.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:01:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.411+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3f68102930 con 0x7f3f4c0b2af0 2024-05-27T07:01:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.425+0000 7f3f5ea00640 1 -- 172.21.15.16:0/2746088916 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3f68102930 con 0x7f3f4c0b2af0 2024-05-27T07:01:16.417 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.427+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 msgr2=0x7f3f4c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.427+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 0x7f3f4c0b5000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f680770d0 tx=0x7f3f58009670 comp rx=0 tx=0).stop 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.427+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 msgr2=0x7f3f681a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.427+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002a9a0 tx=0x7f3f50003c20 comp rx=0 tx=0).stop 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 shutdown_connections 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3f4c0b2af0 0x7f3f4c0b5000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f681102f0 0x7f3f6819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f6810d990 0x7f3f681a2a20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 --2- 172.21.15.16:0/2746088916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f68075480 0x7f3f681a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 >> 172.21.15.16:0/2746088916 conn(0x7f3f68079240 msgr2=0x7f3f68102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 shutdown_connections 2024-05-27T07:01:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:16.428+0000 7f3f67200640 1 -- 172.21.15.16:0/2746088916 wait complete. 2024-05-27T07:01:16.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-27T07:01:16.569 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-27T07:01:16.580 INFO:teuthology.run_tasks:Running task workunit... 2024-05-27T07:01:16.593 INFO:tasks.workunit:Pulling workunits from ref 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:01:16.594 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T07:01:16.594 INFO:tasks.workunit:timeout=3h 2024-05-27T07:01:16.594 INFO:tasks.workunit:cleanup=True 2024-05-27T07:01:16.594 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-27 06:56:57.674566752 +0000 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-27 06:56:57.674566752 +0000 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-27 07:00:52.109992588 +0000 2024-05-27T07:01:16.646 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-27T07:01:16.647 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T07:01:16.647 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T07:01:16.727 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:01:16.773 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T07:01:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:17.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:16 smithi164 ceph-mon[35327]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:01:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:01:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:17.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:16 smithi016 ceph-mon[31415]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:17.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:16 smithi069 ceph-mon[35276]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:01:18.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:17 smithi164 ceph-mon[35327]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:18.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:17 smithi016 ceph-mon[31415]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:18.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:17 smithi069 ceph-mon[35276]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 -- 172.21.15.16:0/2260728446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc01151f0 msgr2=0x7f4dc0117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 --2- 172.21.15.16:0/2260728446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc01151f0 0x7f4dc0117610 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b0a0 tx=0x7f4db800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 -- 172.21.15.16:0/2260728446 shutdown_connections 2024-05-27T07:01:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 --2- 172.21.15.16:0/2260728446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc01151f0 0x7f4dc0117610 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 --2- 172.21.15.16:0/2260728446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc0078df0 0x7f4dc0079200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 --2- 172.21.15.16:0/2260728446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc0114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 -- 172.21.15.16:0/2260728446 >> 172.21.15.16:0/2260728446 conn(0x7f4dc0102070 msgr2=0x7f4dc01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 -- 172.21.15.16:0/2260728446 shutdown_connections 2024-05-27T07:01:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.372+0000 7f4dc8600700 1 -- 172.21.15.16:0/2260728446 wait complete. 2024-05-27T07:01:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.373+0000 7f4dc8600700 1 Processor -- start 2024-05-27T07:01:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 -- start start 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc01151f0 0x7f4dc01a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011a980 con 0x7f4dc0078df0 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011a800 con 0x7f4dc0079740 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.374+0000 7f4dc8600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011ab00 con 0x7f4dc01151f0 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33588/0 (socket says 172.21.15.16:33588) 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 -- 172.21.15.16:0/695991145 learned_addr learned my addr 172.21.15.16:0/695991145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5200700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc6600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc01151f0 0x7f4dc01a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 -- 172.21.15.16:0/695991145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc01151f0 msgr2=0x7f4dc01a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc01151f0 0x7f4dc01a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 -- 172.21.15.16:0/695991145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 msgr2=0x7f4dc01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5c00700 1 -- 172.21.15.16:0/695991145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc01a61a0 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.375+0000 7f4dc5200700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4db400d950 tx=0x7f4db400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dbe000700 1 -- 172.21.15.16:0/695991145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4016070 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dbe000700 1 -- 172.21.15.16:0/695991145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db4010460 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dbe000700 1 -- 172.21.15.16:0/695991145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db400f620 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc01a6430 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.376+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01addb0 con 0x7f4dc0078df0 2024-05-27T07:01:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.380+0000 7f4dbe000700 1 -- 172.21.15.16:0/695991145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f4db4019070 con 0x7f4dc0078df0 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.381+0000 7f4dd7228b80 1 Processor -- start 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.381+0000 7f4dd7228b80 1 -- start start 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dbe000700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 0x7f4da80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dbe000700 1 -- 172.21.15.16:0/695991145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4db40e2900 con 0x7f4dc0078df0 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b1697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b1641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d91b091e40 con 0x55d91b169d30 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d91b0819e0 con 0x55d91b163db0 2024-05-27T07:01:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d91b085a20 con 0x55d91afe4e40 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dc6600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 0x7f4da80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dc5200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dc5200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33598/0 (socket says 172.21.15.16:33598) 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dc5200700 1 -- 172.21.15.16:0/2419018829 learned_addr learned my addr 172.21.15.16:0/2419018829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.382+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b1697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc6600700 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b1641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 -- 172.21.15.16:0/2419018829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 msgr2=0x55d91b1697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b1697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 -- 172.21.15.16:0/2419018829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 msgr2=0x55d91b1641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b1641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 -- 172.21.15.16:0/2419018829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91b04a420 con 0x55d91b169d30 2024-05-27T07:01:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc6600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 0x7f4da80b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db8009dc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4dc5200700 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00dcb0 tx=0x7f4dac00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ebe9e122e16078 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.383+0000 7f4db2a00700 1 -- 172.21.15.16:0/2419018829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00fc90 con 0x55d91b169d30 2024-05-27T07:01:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.384+0000 7f4db2a00700 1 -- 172.21.15.16:0/2419018829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dac00fe30 con 0x55d91b169d30 2024-05-27T07:01:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.384+0000 7f4db2a00700 1 -- 172.21.15.16:0/2419018829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac010630 con 0x55d91b169d30 2024-05-27T07:01:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.384+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2419018829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 msgr2=0x55d91b163870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.384+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00dcb0 tx=0x7f4dac00f360 comp rx=0 tx=0).stop 2024-05-27T07:01:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2419018829 shutdown_connections 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b1641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b1697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2419018829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b163870 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2419018829 >> 172.21.15.16:0/2419018829 conn(0x55d91b044fc0 msgr2=0x55d91b15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2419018829 shutdown_connections 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2419018829 wait complete. 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.385+0000 7f4dd7228b80 1 -- start start 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 Processor -- start 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 0x55d91b16d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d91af87fa0 con 0x55d91b169d30 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d91b022760 con 0x55d91afe4e40 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.386+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d91b170800 con 0x55d91b163db0 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dd7228b80 10 client.0 _ll_register_callbacks cb 0x55d91b02bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33614/0 (socket says 172.21.15.16:33614) 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 -- 172.21.15.16:0/2860086186 learned_addr learned my addr 172.21.15.16:0/2860086186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 0x55d91b16d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc5200700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 -- 172.21.15.16:0/2860086186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 msgr2=0x55d91b16d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 -- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 msgr2=0x55d91b16d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 0x55d91b16d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc6600700 1 -- 172.21.15.16:0/2860086186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91afd18d0 con 0x55d91b169d30 2024-05-27T07:01:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.387+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 0x55d91b16d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4dc6600700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004e70 tx=0x7f4db800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44810f105804d77d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4da7400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8005040 con 0x55d91b169d30 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dac00fe30 con 0x55d91b169d30 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4db80051e0 con 0x55d91b169d30 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4db80053e0 con 0x55d91b169d30 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4dd7228b80 10 client.15510 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.388+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d91b052060 con 0x55d91b169d30 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dac00fe30 con 0x55d91b169d30 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4db8011070 con 0x55d91b169d30 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 20 client.15510 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b17d600 0x55d91b17fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 4 client.15510 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.389+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d91b093d90 con 0x55d91b17d600 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.390+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b17d600 0x55d91b17fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.391+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b17d600 0x55d91b17fa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0d5ffadffb8bee server_cookie=91e29ff8a360c884 in_seq=0 out_seq=0 2024-05-27T07:01:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.391+0000 7f4da7400700 10 client.15510 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.393+0000 7f4da7400700 1 -- 172.21.15.16:0/2860086186 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d91b093d90 con 0x55d91b17d600 2024-05-27T07:01:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.393+0000 7f4da7400700 10 client.15510 handle_command_reply: tid=0 2024-05-27T07:01:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.393+0000 7f4dd7228b80 1 client.15510 shutdown 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.393+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 msgr2=0x55d91b16dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.393+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004e70 tx=0x7f4db800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 shutdown_connections 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b17d600 0x55d91b17fa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b16dea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91afe4e40 0x55d91b16d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/2860086186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4da7400700 0 client.15510 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 >> 172.21.15.16:0/2860086186 conn(0x55d91b044fc0 msgr2=0x55d91b15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 shutdown_connections 2024-05-27T07:01:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 1 -- 172.21.15.16:0/2860086186 wait complete. 2024-05-27T07:01:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.394+0000 7f4dd7228b80 20 client.15510 trim_cache size 0 max 16384 2024-05-27T07:01:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.416+0000 7f4dd7228b80 1 Processor -- start 2024-05-27T07:01:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 -- start start 2024-05-27T07:01:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b169d30 0x55d91b179860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b6a70 con 0x55d91afe4e40 2024-05-27T07:01:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b85b0 con 0x55d91b169d30 2024-05-27T07:01:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b7140 con 0x55d91b163db0 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.417+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33628/0 (socket says 172.21.15.16:33628) 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 -- 172.21.15.16:0/3745241338 learned_addr learned my addr 172.21.15.16:0/3745241338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc6600700 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b169d30 0x55d91b179860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5200700 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 -- 172.21.15.16:0/3745241338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 msgr2=0x55d91b16c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 -- 172.21.15.16:0/3745241338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b169d30 msgr2=0x55d91b179860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b169d30 0x55d91b179860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 -- 172.21.15.16:0/3745241338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91b16fcc0 con 0x55d91afe4e40 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.418+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e010 tx=0x7f4db4091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c879a2c231980e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.419+0000 7f4db2000700 1 -- 172.21.15.16:0/3745241338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db40c2070 con 0x55d91afe4e40 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.419+0000 7f4db2000700 1 -- 172.21.15.16:0/3745241338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db40910b0 con 0x55d91afe4e40 2024-05-27T07:01:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.419+0000 7f4db2000700 1 -- 172.21.15.16:0/3745241338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4020040 con 0x55d91afe4e40 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 -- 172.21.15.16:0/3745241338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 msgr2=0x55d91b163870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e010 tx=0x7f4db4091040 comp rx=0 tx=0).stop 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 -- 172.21.15.16:0/3745241338 shutdown_connections 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b169d30 0x55d91b179860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91afe4e40 0x55d91b163870 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/3745241338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91b163db0 0x55d91b16c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 -- 172.21.15.16:0/3745241338 >> 172.21.15.16:0/3745241338 conn(0x55d91b044fc0 msgr2=0x55d91b16d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 -- 172.21.15.16:0/3745241338 shutdown_connections 2024-05-27T07:01:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.420+0000 7f4dd7228b80 1 -- 172.21.15.16:0/3745241338 wait complete. 2024-05-27T07:01:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 -- start start 2024-05-27T07:01:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 Processor -- start 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b176160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b184a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b7ee0 con 0x55d91b169d30 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b7810 con 0x55d91b163db0 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.421+0000 7f4dd7228b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01b4a10 con 0x55d91afe4e40 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dd7228b80 10 client.0 _ll_register_callbacks cb 0x55d91b02bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33644/0 (socket says 172.21.15.16:33644) 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 -- 172.21.15.16:0/1783234934 learned_addr learned my addr 172.21.15.16:0/1783234934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b176160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc5200700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b184a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 -- 172.21.15.16:0/1783234934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 msgr2=0x55d91b176160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b176160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 -- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 msgr2=0x55d91b184a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b184a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc6600700 1 -- 172.21.15.16:0/1783234934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91b185650 con 0x55d91b169d30 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.422+0000 7f4dc5200700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b184a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.423+0000 7f4dc6600700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4db801db70 tx=0x7f4db80079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72fe034f394b76a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.423+0000 7f4da7400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.423+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8005210 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.423+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db40910b0 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.423+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4db8005c60 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4db8045050 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 10 client.15516 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d91b1e7fb0 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4db40910b0 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4db8011040 con 0x55d91b169d30 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 20 client.15516 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b1e8630 0x55d91b1eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 4 client.15516 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:01:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.424+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d91b08d570 con 0x55d91b1e8630 2024-05-27T07:01:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.425+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b1e8630 0x55d91b1eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.426+0000 7f4dc5c00700 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b1e8630 0x55d91b1eaa50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1deb758fa3390bf8 server_cookie=326770dc80a82c3d in_seq=0 out_seq=0 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.426+0000 7f4da7400700 10 client.15516 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4da7400700 1 -- 172.21.15.16:0/1783234934 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d91b08d570 con 0x55d91b1e8630 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4da7400700 10 client.15516 handle_command_reply: tid=0 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 client.15516 shutdown 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 msgr2=0x55d91b184f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4db801db70 tx=0x7f4db80079e0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 shutdown_connections 2024-05-27T07:01:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d91b1e8630 0x55d91b1eaa50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d91b169d30 0x55d91b184f50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d91afe4e40 0x55d91b176160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 --2- 172.21.15.16:0/1783234934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d91b163db0 0x55d91b184a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 >> 172.21.15.16:0/1783234934 conn(0x55d91b044fc0 msgr2=0x55d91b16c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4da7400700 0 client.15516 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 shutdown_connections 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 1 -- 172.21.15.16:0/1783234934 wait complete. 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.427+0000 7f4dd7228b80 20 client.15516 trim_cache size 0 max 16384 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:01:19.421 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: "6464d79d-55f0-4170-94d8-d95a960d4bfc": { 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "6464d79d-55f0-4170-94d8-d95a960d4bfc", 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:01:19.422 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 msgr2=0x7f4da80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 0x7f4da80b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db8009dc0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 msgr2=0x7f4dc01a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4db400d950 tx=0x7f4db400dd10 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 shutdown_connections 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f4da80b2b10 0x7f4da80b5020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dc01151f0 0x7f4dc01a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0078df0 0x7f4dc01a12d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 --2- 172.21.15.16:0/695991145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0079740 0x7f4dc01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 >> 172.21.15.16:0/695991145 conn(0x7f4dc0102070 msgr2=0x7f4dc007c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.428+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 shutdown_connections 2024-05-27T07:01:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:19.429+0000 7f4dc8600700 1 -- 172.21.15.16:0/695991145 wait complete. 2024-05-27T07:01:20.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:19 smithi164 ceph-mon[35327]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:20.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:19 smithi016 ceph-mon[31415]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:20.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:19 smithi069 ceph-mon[35276]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:01:20.374 INFO:tasks.cephfs.filesystem:scrub status for tag:6464d79d-55f0-4170-94d8-d95a960d4bfc - {'path': '/', 'tag': '6464d79d-55f0-4170-94d8-d95a960d4bfc', 'options': 'recursive,force'} 2024-05-27T07:01:22.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:21 smithi164 ceph-mon[35327]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-27T07:01:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:21 smithi016 ceph-mon[31415]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-27T07:01:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:21 smithi069 ceph-mon[35276]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-27T07:01:24.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:23 smithi164 ceph-mon[35327]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-27T07:01:24.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:23 smithi016 ceph-mon[31415]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-27T07:01:24.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:23 smithi069 ceph-mon[35276]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-27T07:01:26.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:25 smithi164 ceph-mon[35327]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:01:26.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:25 smithi016 ceph-mon[31415]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:01:26.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:25 smithi069 ceph-mon[35276]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:01:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:27 smithi164 ceph-mon[35327]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:27 smithi016 ceph-mon[31415]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:28.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:27 smithi069 ceph-mon[35276]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:29.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:28 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:01:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:28 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:01:29.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:28 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:01:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:29 smithi069 ceph-mon[35276]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:30.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:29 smithi164 ceph-mon[35327]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:30.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:29 smithi016 ceph-mon[31415]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-27T07:01:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:31 smithi164 ceph-mon[35327]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-05-27T07:01:32.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:31 smithi016 ceph-mon[31415]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-05-27T07:01:32.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:31 smithi069 ceph-mon[35276]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-05-27T07:01:34.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:33 smithi164 ceph-mon[35327]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-27T07:01:34.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:33 smithi016 ceph-mon[31415]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-27T07:01:34.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:33 smithi069 ceph-mon[35276]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-27T07:01:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:35 smithi164 ceph-mon[35327]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:01:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:35 smithi016 ceph-mon[31415]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:01:36.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:35 smithi069 ceph-mon[35276]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:01:38.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:37 smithi164 ceph-mon[35327]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:38.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:37 smithi016 ceph-mon[31415]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:38.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:37 smithi069 ceph-mon[35276]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:41.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:40 smithi164 ceph-mon[35327]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:41.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:40 smithi016 ceph-mon[31415]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:41.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:40 smithi069 ceph-mon[35276]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:43.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:42 smithi164 ceph-mon[35327]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:43.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:42 smithi016 ceph-mon[31415]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:43.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:42 smithi069 ceph-mon[35276]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:45.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:44 smithi164 ceph-mon[35327]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:45.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:44 smithi016 ceph-mon[31415]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:44 smithi069 ceph-mon[35276]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:47.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:46 smithi164 ceph-mon[35327]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:46 smithi016 ceph-mon[31415]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:47.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:46 smithi069 ceph-mon[35276]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:49.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:48 smithi164 ceph-mon[35327]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:49.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:48 smithi016 ceph-mon[31415]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:49.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:48 smithi069 ceph-mon[35276]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:50.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:51.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:50 smithi016 ceph-mon[31415]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:01:51.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:50 smithi164 ceph-mon[35327]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:01:51.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:50 smithi069 ceph-mon[35276]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:01:53.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:52 smithi016 ceph-mon[31415]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:53.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:52 smithi164 ceph-mon[35327]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:53.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:52 smithi069 ceph-mon[35276]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 -- 172.21.15.16:0/594265195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410b2d0 msgr2=0x7f8a44117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 --2- 172.21.15.16:0/594265195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410b2d0 0x7f8a44117800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b0a0 tx=0x7f8a3c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 -- 172.21.15.16:0/594265195 shutdown_connections 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 --2- 172.21.15.16:0/594265195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410b2d0 0x7f8a44117800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 --2- 172.21.15.16:0/594265195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a44110960 0x7f8a44110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 --2- 172.21.15.16:0/594265195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a4410a980 0x7f8a4410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 -- 172.21.15.16:0/594265195 >> 172.21.15.16:0/594265195 conn(0x7f8a44077c70 msgr2=0x7f8a440780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 -- 172.21.15.16:0/594265195 shutdown_connections 2024-05-27T07:01:53.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.301+0000 7f8a49e00700 1 -- 172.21.15.16:0/594265195 wait complete. 2024-05-27T07:01:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.302+0000 7f8a49e00700 1 Processor -- start 2024-05-27T07:01:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.302+0000 7f8a49e00700 1 -- start start 2024-05-27T07:01:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4410b2d0 0x7f8a441a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a44110960 0x7f8a441a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4411a980 con 0x7f8a4410a980 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4411a800 con 0x7f8a44110960 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a49e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4411ab00 con 0x7f8a4410b2d0 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59022/0 (socket says 172.21.15.16:59022) 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a42a00700 1 -- 172.21.15.16:0/1216587822 learned_addr learned my addr 172.21.15.16:0/1216587822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a42000700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4410b2d0 0x7f8a441a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.303+0000 7f8a43400700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a44110960 0x7f8a441a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 -- 172.21.15.16:0/1216587822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4410b2d0 msgr2=0x7f8a441a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4410b2d0 0x7f8a441a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 -- 172.21.15.16:0/1216587822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a44110960 msgr2=0x7f8a441a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a44110960 0x7f8a441a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 -- 172.21.15.16:0/1216587822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a441a6220 con 0x7f8a4410a980 2024-05-27T07:01:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a42a00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.304+0000 7f8a48a00700 1 -- 172.21.15.16:0/1216587822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34017070 con 0x7f8a4410a980 2024-05-27T07:01:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.305+0000 7f8a48a00700 1 -- 172.21.15.16:0/1216587822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a34010460 con 0x7f8a4410a980 2024-05-27T07:01:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.305+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a441a6510 con 0x7f8a4410a980 2024-05-27T07:01:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.305+0000 7f8a48a00700 1 -- 172.21.15.16:0/1216587822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400de60 con 0x7f8a4410a980 2024-05-27T07:01:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.305+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441ade30 con 0x7f8a4410a980 2024-05-27T07:01:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.309+0000 7f8a48a00700 1 -- 172.21.15.16:0/1216587822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7f8a34010610 con 0x7f8a4410a980 2024-05-27T07:01:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.310+0000 7f8a58888b80 1 Processor -- start 2024-05-27T07:01:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.310+0000 7f8a58888b80 1 -- start start 2024-05-27T07:01:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.310+0000 7f8a48a00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 0x7f8a280b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x5632531377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253137d30 0x563253131870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56325305fe40 con 0x563253131db0 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56325304f9e0 con 0x563253137d30 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a58888b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563253053a20 con 0x563252fb2e40 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a48a00700 1 -- 172.21.15.16:0/1216587822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a340e27a0 con 0x7f8a4410a980 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59034/0 (socket says 172.21.15.16:59034) 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a43400700 1 -- 172.21.15.16:0/2673711552 learned_addr learned my addr 172.21.15.16:0/2673711552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a42a00700 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x5632531377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.311+0000 7f8a43400700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 0x7f8a280b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a42000700 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253137d30 0x563253131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a43400700 1 -- 172.21.15.16:0/2673711552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 msgr2=0x5632531377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a43400700 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x5632531377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a43400700 1 -- 172.21.15.16:0/2673711552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253137d30 msgr2=0x563253131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a43400700 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253137d30 0x563253131870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.312+0000 7f8a43400700 1 -- 172.21.15.16:0/2673711552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563253018420 con 0x563253131db0 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.313+0000 7f8a43400700 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009610 tx=0x7f8a3c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf83cbe46dcbc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.313+0000 7f8a43400700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 0x7f8a280b5020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004cd0 tx=0x7f8a3c013000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.313+0000 7f8a38c00700 1 -- 172.21.15.16:0/2673711552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c02d070 con 0x563253131db0 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.313+0000 7f8a38c00700 1 -- 172.21.15.16:0/2673711552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3c009410 con 0x563253131db0 2024-05-27T07:01:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.313+0000 7f8a38c00700 1 -- 172.21.15.16:0/2673711552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c01fd10 con 0x563253131db0 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 -- 172.21.15.16:0/2673711552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 msgr2=0x5632531321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009610 tx=0x7f8a3c004c60 comp rx=0 tx=0).stop 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 -- 172.21.15.16:0/2673711552 shutdown_connections 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x5632531321c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x5632531377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2673711552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253137d30 0x563253131870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 -- 172.21.15.16:0/2673711552 >> 172.21.15.16:0/2673711552 conn(0x563253012fc0 msgr2=0x56325312c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 -- 172.21.15.16:0/2673711552 shutdown_connections 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.314+0000 7f8a58888b80 1 -- 172.21.15.16:0/2673711552 wait complete. 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.315+0000 7f8a58888b80 1 -- start start 2024-05-27T07:01:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.315+0000 7f8a58888b80 1 Processor -- start 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325313b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253131db0 0x56325313b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563252f55fa0 con 0x563253137d30 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563252ff0760 con 0x563252fb2e40 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56325313e7a0 con 0x563253131db0 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a58888b80 10 client.0 _ll_register_callbacks cb 0x563252ff9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.316+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59040/0 (socket says 172.21.15.16:59040) 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 -- 172.21.15.16:0/2420359728 learned_addr learned my addr 172.21.15.16:0/2420359728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a42000700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253131db0 0x56325313b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a42a00700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325313b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 -- 172.21.15.16:0/2420359728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253131db0 msgr2=0x56325313b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253131db0 0x56325313b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 -- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 msgr2=0x56325313b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325313b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.317+0000 7f8a43400700 1 -- 172.21.15.16:0/2420359728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563252f9f8d0 con 0x563253137d30 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a43400700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009a20 tx=0x7f8a3c01fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff43130d1c6136f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c02d070 con 0x563253137d30 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a3c029620 con 0x563253137d30 2024-05-27T07:01:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c009410 con 0x563253137d30 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a3c040050 con 0x563253137d30 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a58888b80 10 client.15528 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.318+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563253020060 con 0x563253137d30 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3c009410 con 0x563253137d30 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3c03b030 con 0x563253137d30 2024-05-27T07:01:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 20 client.15528 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56325314b470 0x56325314d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 4 client.15528 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:01:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.319+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563253061d90 con 0x56325314b470 2024-05-27T07:01:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.320+0000 7f8a42a00700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56325314b470 0x56325314d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.321+0000 7f8a42a00700 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56325314b470 0x56325314d890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f507e5c2dedf00d8 server_cookie=9defe656ea3987b2 in_seq=0 out_seq=0 2024-05-27T07:01:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.321+0000 7f8a2d600700 10 client.15528 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a2d600700 1 -- 172.21.15.16:0/2420359728 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563253061d90 con 0x56325314b470 2024-05-27T07:01:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a2d600700 10 client.15528 handle_command_reply: tid=0 2024-05-27T07:01:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 client.15528 shutdown 2024-05-27T07:01:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 msgr2=0x56325313bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009a20 tx=0x7f8a3c01fd50 comp rx=0 tx=0).stop 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 shutdown_connections 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56325314b470 0x56325314d890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313bed0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325313b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 --2- 172.21.15.16:0/2420359728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253131db0 0x56325313b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 >> 172.21.15.16:0/2420359728 conn(0x563253012fc0 msgr2=0x56325312c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.326+0000 7f8a2d600700 0 client.15528 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.327+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 shutdown_connections 2024-05-27T07:01:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.327+0000 7f8a58888b80 1 -- 172.21.15.16:0/2420359728 wait complete. 2024-05-27T07:01:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.327+0000 7f8a58888b80 20 client.15528 trim_cache size 0 max 16384 2024-05-27T07:01:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.350+0000 7f8a58888b80 1 Processor -- start 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.350+0000 7f8a58888b80 1 -- start start 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x563253131870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 0x563253147a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b69c0 con 0x563253137d30 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b8500 con 0x563253131db0 2024-05-27T07:01:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a58888b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b7090 con 0x563252fb2e40 2024-05-27T07:01:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a43400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59044/0 (socket says 172.21.15.16:59044) 2024-05-27T07:01:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.351+0000 7f8a43400700 1 -- 172.21.15.16:0/3943868801 learned_addr learned my addr 172.21.15.16:0/3943868801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a42a00700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x563253131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a42000700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 0x563253147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 -- 172.21.15.16:0/3943868801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 msgr2=0x563253131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x563253131870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 -- 172.21.15.16:0/3943868801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 msgr2=0x563253147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 0x563253147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 -- 172.21.15.16:0/3943868801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325313e400 con 0x563253137d30 2024-05-27T07:01:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a42000700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 0x563253147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:01:53.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.352+0000 7f8a43400700 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d010 tx=0x7f8a3c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a801fe9f95fbd8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.353+0000 7f8a2fe00700 1 -- 172.21.15.16:0/3943868801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c049070 con 0x563253137d30 2024-05-27T07:01:53.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.353+0000 7f8a2fe00700 1 -- 172.21.15.16:0/3943868801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3c009410 con 0x563253137d30 2024-05-27T07:01:53.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.353+0000 7f8a2fe00700 1 -- 172.21.15.16:0/3943868801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c02d070 con 0x563253137d30 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 -- 172.21.15.16:0/3943868801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 msgr2=0x56325313a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d010 tx=0x7f8a3c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 -- 172.21.15.16:0/3943868801 shutdown_connections 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253137d30 0x56325313a770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563252fb2e40 0x563253131870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 --2- 172.21.15.16:0/3943868801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563253131db0 0x563253147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 -- 172.21.15.16:0/3943868801 >> 172.21.15.16:0/3943868801 conn(0x563253012fc0 msgr2=0x56325313b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 -- 172.21.15.16:0/3943868801 shutdown_connections 2024-05-27T07:01:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.354+0000 7f8a58888b80 1 -- 172.21.15.16:0/3943868801 wait complete. 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.355+0000 7f8a58888b80 1 -- start start 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.355+0000 7f8a58888b80 1 Processor -- start 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.355+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325314e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.355+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a58888b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253137d30 0x56325314f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a58888b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b7e30 con 0x563253131db0 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a58888b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b7760 con 0x563252fb2e40 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a58888b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441b4960 con 0x563253137d30 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a58888b80 10 client.0 _ll_register_callbacks cb 0x563252ff9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59058/0 (socket says 172.21.15.16:59058) 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 -- 172.21.15.16:0/15634797 learned_addr learned my addr 172.21.15.16:0/15634797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42a00700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325314e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a43400700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253137d30 0x56325314f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 -- 172.21.15.16:0/15634797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253137d30 msgr2=0x56325314f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253137d30 0x56325314f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 -- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 msgr2=0x56325314e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325314e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.356+0000 7f8a42000700 1 -- 172.21.15.16:0/15634797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632531b4ce0 con 0x563253131db0 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a42000700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a4406d110 tx=0x7f8a30011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b360d67c67cd819 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:01:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30021060 con 0x563253131db0 2024-05-27T07:01:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a30004440 con 0x563253131db0 2024-05-27T07:01:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c009410 con 0x563253131db0 2024-05-27T07:01:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a30028050 con 0x563253131db0 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a58888b80 10 client.15534 fetch_fsmap learned FSMap version 31 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.357+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5632531b5f60 con 0x563253131db0 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3c009410 con 0x563253131db0 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3001b030 con 0x563253131db0 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 10 client.15534 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 20 client.15534 populate_metadata read hostname 'smithi016' 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5632531b66d0 0x5632531b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 4 client.15534 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.358+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56325305b570 con 0x5632531b66d0 2024-05-27T07:01:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.359+0000 7f8a42a00700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5632531b66d0 0x5632531b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:53.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a42a00700 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5632531b66d0 0x5632531b8af0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29c4b0f9b996561 server_cookie=61271fb99a8cc405 in_seq=0 out_seq=0 2024-05-27T07:01:53.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a2d600700 10 client.15534 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a2d600700 1 -- 172.21.15.16:0/15634797 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56325305b570 con 0x5632531b66d0 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a2d600700 10 client.15534 handle_command_reply: tid=0 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a58888b80 1 client.15534 shutdown 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 msgr2=0x56325314eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.360+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a4406d110 tx=0x7f8a30011720 comp rx=0 tx=0).stop 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 shutdown_connections 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5632531b66d0 0x5632531b8af0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563253137d30 0x56325314f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563252fb2e40 0x56325314e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 --2- 172.21.15.16:0/15634797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563253131db0 0x56325314eb90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 >> 172.21.15.16:0/15634797 conn(0x563253012fc0 msgr2=0x56325313a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a2d600700 0 client.15534 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 shutdown_connections 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 1 -- 172.21.15.16:0/15634797 wait complete. 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.361+0000 7f8a58888b80 20 client.15534 trim_cache size 0 max 16384 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:01:53.355 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 msgr2=0x7f8a280b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 0x7f8a280b5020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004cd0 tx=0x7f8a3c013000 comp rx=0 tx=0).stop 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 msgr2=0x7f8a441a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dd40 comp rx=0 tx=0).stop 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 shutdown_connections 2024-05-27T07:01:53.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a280b2b10 0x7f8a280b5020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a44110960 0x7f8a441a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410a980 0x7f8a441a1350 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 --2- 172.21.15.16:0/1216587822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4410b2d0 0x7f8a441a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 >> 172.21.15.16:0/1216587822 conn(0x7f8a44077c70 msgr2=0x7f8a4407b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 shutdown_connections 2024-05-27T07:01:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:53.362+0000 7f8a49e00700 1 -- 172.21.15.16:0/1216587822 wait complete. 2024-05-27T07:01:54.393 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:01:54.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:01:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:55.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:54 smithi016 ceph-mon[31415]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:55.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:54 smithi164 ceph-mon[35327]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:55.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:54 smithi069 ceph-mon[35276]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:56.979 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:56 smithi016 ceph-mon[31415]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:57.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:56 smithi164 ceph-mon[35327]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:57.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:56 smithi069 ceph-mon[35276]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:01:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 -- 172.21.15.16:0/3095645156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 msgr2=0x7fad201111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 --2- 172.21.15.16:0/3095645156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201111d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b0d0 tx=0x7fad1800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 -- 172.21.15.16:0/3095645156 shutdown_connections 2024-05-27T07:01:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 --2- 172.21.15.16:0/3095645156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201111d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 --2- 172.21.15.16:0/3095645156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 0x7fad20104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 --2- 172.21.15.16:0/3095645156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 0x7fad20109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 -- 172.21.15.16:0/3095645156 >> 172.21.15.16:0/3095645156 conn(0x7fad20079ef0 msgr2=0x7fad2007a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.374+0000 7fad26200700 1 -- 172.21.15.16:0/3095645156 shutdown_connections 2024-05-27T07:01:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.375+0000 7fad26200700 1 -- 172.21.15.16:0/3095645156 wait complete. 2024-05-27T07:01:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 Processor -- start 2024-05-27T07:01:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 -- start start 2024-05-27T07:01:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 0x7fad2019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 0x7fad2019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad201143a0 con 0x7fad20109870 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad20114220 con 0x7fad20108f20 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.376+0000 7fad26200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad20114520 con 0x7fad20104640 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.377+0000 7fad1fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.377+0000 7fad1fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59084/0 (socket says 172.21.15.16:59084) 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.377+0000 7fad1fe00700 1 -- 172.21.15.16:0/931170451 learned_addr learned my addr 172.21.15.16:0/931170451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.377+0000 7fad1f400700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 0x7fad2019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.377+0000 7fad1ea00700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 0x7fad2019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 -- 172.21.15.16:0/931170451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 msgr2=0x7fad2019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 0x7fad2019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 -- 172.21.15.16:0/931170451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 msgr2=0x7fad2019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 0x7fad2019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 -- 172.21.15.16:0/931170451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad201a1e00 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad1fe00700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fad18007d10 tx=0x7fad18007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1801f070 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad18003e10 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad201a2090 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.378+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1801b720 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.379+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201a2570 con 0x7fad20109870 2024-05-27T07:01:57.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.380+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad200512b0 con 0x7fad20109870 2024-05-27T07:01:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.382+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fad18022070 con 0x7fad20109870 2024-05-27T07:01:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.385+0000 7fad17e00700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 0x7fad040b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:01:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.385+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fad1800f020 con 0x7fad20109870 2024-05-27T07:01:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.385+0000 7fad1f400700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 0x7fad040b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:01:57.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.386+0000 7fad1f400700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 0x7fad040b5020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad10007be0 tx=0x7fad100073d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:01:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.388+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad18029020 con 0x7fad20109870 2024-05-27T07:01:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.678+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad201a9ad0 con 0x7fad20109870 2024-05-27T07:01:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.680+0000 7fad17e00700 1 -- 172.21.15.16:0/931170451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7fad180ab8f0 con 0x7fad20109870 2024-05-27T07:01:57.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:01:57.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:01:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.683+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 msgr2=0x7fad040b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.683+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 0x7fad040b5020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad10007be0 tx=0x7fad100073d0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.683+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 msgr2=0x7fad201a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:01:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.683+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fad18007d10 tx=0x7fad18007d40 comp rx=0 tx=0).stop 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 shutdown_connections 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fad040b2b10 0x7fad040b5020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20109870 0x7fad201a1760 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20104640 0x7fad2019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 --2- 172.21.15.16:0/931170451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad20108f20 0x7fad2019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 >> 172.21.15.16:0/931170451 conn(0x7fad20079ef0 msgr2=0x7fad2010f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 shutdown_connections 2024-05-27T07:01:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:01:57.684+0000 7fad26200700 1 -- 172.21.15.16:0/931170451 wait complete. 2024-05-27T07:01:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:01:58.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:57 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/931170451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:58.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:57 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/931170451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:58.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:57 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/931170451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:01:58.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:01:59.124 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:01:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:01:59 smithi016 ceph-mon[31415]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:59.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:01:59 smithi164 ceph-mon[35327]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:01:59.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:01:59 smithi069 ceph-mon[35276]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:00.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:02:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:02:00.286+0000 7f2a18a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:02:00.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:02:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:02:00.286+0000 7f2a18a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:02:00.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:02:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:02:00.286+0000 7f2a18a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:02:00.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:02:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:02:00.286+0000 7f2a18a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:02:00.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:02:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:02:00.286+0000 7f2a18a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:02:01.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:01 smithi016 ceph-mon[31415]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 -- 172.21.15.16:0/3759770384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c075ad0 msgr2=0x7fec4c075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 --2- 172.21.15.16:0/3759770384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c075ad0 0x7fec4c075ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009990 tx=0x7fec3c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 -- 172.21.15.16:0/3759770384 shutdown_connections 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 --2- 172.21.15.16:0/3759770384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c110c10 0x7fec4c113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 --2- 172.21.15.16:0/3759770384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c075ad0 0x7fec4c075ee0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 --2- 172.21.15.16:0/3759770384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c076420 0x7fec4c1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 -- 172.21.15.16:0/3759770384 >> 172.21.15.16:0/3759770384 conn(0x7fec4c100030 msgr2=0x7fec4c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 -- 172.21.15.16:0/3759770384 shutdown_connections 2024-05-27T07:02:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.705+0000 7fec51a00700 1 -- 172.21.15.16:0/3759770384 wait complete. 2024-05-27T07:02:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.706+0000 7fec51a00700 1 Processor -- start 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.706+0000 7fec51a00700 1 -- start start 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 0x7fec4c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c076420 0x7fec4c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c116590 con 0x7fec4c110c10 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c116410 con 0x7fec4c075ad0 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec51a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c116710 con 0x7fec4c076420 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59126/0 (socket says 172.21.15.16:59126) 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.707+0000 7fec4b400700 1 -- 172.21.15.16:0/2345968141 learned_addr learned my addr 172.21.15.16:0/2345968141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4a000700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c076420 0x7fec4c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4aa00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 0x7fec4c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 -- 172.21.15.16:0/2345968141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c076420 msgr2=0x7fec4c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c076420 0x7fec4c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 -- 172.21.15.16:0/2345968141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 msgr2=0x7fec4c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 0x7fec4c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 -- 172.21.15.16:0/2345968141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1a1de0 con 0x7fec4c110c10 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4aa00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 0x7fec4c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.708+0000 7fec4b400700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec44007d70 tx=0x7fec4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.709+0000 7fec43400700 1 -- 172.21.15.16:0/2345968141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44016020 con 0x7fec4c110c10 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.709+0000 7fec43400700 1 -- 172.21.15.16:0/2345968141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec4400ad40 con 0x7fec4c110c10 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.709+0000 7fec43400700 1 -- 172.21.15.16:0/2345968141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440115e0 con 0x7fec4c110c10 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.709+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c1a20d0 con 0x7fec4c110c10 2024-05-27T07:02:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.709+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1a2610 con 0x7fec4c110c10 2024-05-27T07:02:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.713+0000 7fec43400700 1 -- 172.21.15.16:0/2345968141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 149879+0+0 (secure 0 0 0) 0x7fec44019020 con 0x7fec4c110c10 2024-05-27T07:02:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.714+0000 7fec60509b80 1 Processor -- start 2024-05-27T07:02:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.714+0000 7fec60509b80 1 -- start start 2024-05-27T07:02:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec43400700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 0x7fec300b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8289e59f0 0x55a828a78330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8288f9e40 0x55a828a78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8289a6e40 con 0x55a828a78e70 2024-05-27T07:02:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8289969e0 con 0x55a8288f9e40 2024-05-27T07:02:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec60509b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a82899aa20 con 0x55a8289e59f0 2024-05-27T07:02:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec43400700 1 -- 172.21.15.16:0/2345968141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fec440e2a60 con 0x7fec4c110c10 2024-05-27T07:02:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec4a000700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 0x7fec300b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59132/0 (socket says 172.21.15.16:59132) 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec4b400700 1 -- 172.21.15.16:0/3412610634 learned_addr learned my addr 172.21.15.16:0/3412610634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.715+0000 7fec4aa00700 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8289e59f0 0x55a828a78330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4a000700 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8288f9e40 0x55a828a78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 -- 172.21.15.16:0/3412610634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8289e59f0 msgr2=0x55a828a78330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8289e59f0 0x55a828a78330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 -- 172.21.15.16:0/3412610634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8288f9e40 msgr2=0x55a828a78870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8288f9e40 0x55a828a78870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 -- 172.21.15.16:0/3412610634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82895f420 con 0x55a828a78e70 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4a000700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 0x7fec300b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec3800b6c0 tx=0x7fec38012890 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.716+0000 7fec4b400700 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec440a5890 tx=0x7fec440a9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ac0ab16f34409d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.717+0000 7fec37e00700 1 -- 172.21.15.16:0/3412610634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440a9320 con 0x55a828a78e70 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.717+0000 7fec37e00700 1 -- 172.21.15.16:0/3412610634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.717+0000 7fec37e00700 1 -- 172.21.15.16:0/3412610634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440dff20 con 0x55a828a78e70 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 -- 172.21.15.16:0/3412610634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 msgr2=0x55a828a85110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec440a5890 tx=0x7fec440a9230 comp rx=0 tx=0).stop 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 -- 172.21.15.16:0/3412610634 shutdown_connections 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a85110 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8289e59f0 0x55a828a78330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 --2- 172.21.15.16:0/3412610634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8288f9e40 0x55a828a78870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 -- 172.21.15.16:0/3412610634 >> 172.21.15.16:0/3412610634 conn(0x55a828993390 msgr2=0x55a828a744c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 -- 172.21.15.16:0/3412610634 shutdown_connections 2024-05-27T07:02:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.718+0000 7fec60509b80 1 -- 172.21.15.16:0/3412610634 wait complete. 2024-05-27T07:02:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.719+0000 7fec60509b80 1 -- start start 2024-05-27T07:02:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.719+0000 7fec60509b80 1 Processor -- start 2024-05-27T07:02:01.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.719+0000 7fec60509b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a82640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.719+0000 7fec60509b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec60509b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec60509b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a82889cfa0 con 0x55a828a78e70 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec60509b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a828937760 con 0x55a8289e59f0 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec60509b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a828a85ac0 con 0x55a8288f9e40 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec60509b80 10 client.0 _ll_register_callbacks cb 0x55a828940c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59148/0 (socket says 172.21.15.16:59148) 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 -- 172.21.15.16:0/158298316 learned_addr learned my addr 172.21.15.16:0/158298316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4a000700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a82b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4aa00700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a82640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 -- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 msgr2=0x55a828a82640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a82640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 -- 172.21.15.16:0/158298316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 msgr2=0x55a828a82b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a82b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.720+0000 7fec4b400700 1 -- 172.21.15.16:0/158298316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8288e68d0 con 0x55a828a78e70 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec4aa00700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec4b400700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec44005c30 tx=0x7fec440b2490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c6915ed3cafefe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec34c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440a9320 con 0x55a828a78e70 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fec440a5e00 con 0x55a828a78e70 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.721+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fec440a9810 con 0x55a828a78e70 2024-05-27T07:02:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 10 client.15558 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a828967060 con 0x55a828a78e70 2024-05-27T07:02:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fec440a8a70 con 0x55a828a78e70 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 10 client.15558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 20 client.15558 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828a92890 0x55a828a94cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 4 client.15558 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.722+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8289a8d90 con 0x55a828a92890 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.723+0000 7fec4aa00700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828a92890 0x55a828a94cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.724+0000 7fec4aa00700 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828a92890 0x55a828a94cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b903131a0dac9e7b server_cookie=74b851ba0f1b8dbc in_seq=0 out_seq=0 2024-05-27T07:02:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.724+0000 7fec34c00700 10 client.15558 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec34c00700 1 -- 172.21.15.16:0/158298316 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8289a8d90 con 0x55a828a92890 2024-05-27T07:02:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec34c00700 10 client.15558 handle_command_reply: tid=0 2024-05-27T07:02:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 client.15558 shutdown 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 msgr2=0x55a828a830c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec44005c30 tx=0x7fec440b2490 comp rx=0 tx=0).stop 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 shutdown_connections 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828a92890 0x55a828a94cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a830c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 --2- 172.21.15.16:0/158298316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 >> 172.21.15.16:0/158298316 conn(0x55a828993390 msgr2=0x55a828a78310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec34c00700 0 client.15558 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.726+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 shutdown_connections 2024-05-27T07:02:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.727+0000 7fec60509b80 1 -- 172.21.15.16:0/158298316 wait complete. 2024-05-27T07:02:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.727+0000 7fec60509b80 20 client.15558 trim_cache size 0 max 16384 2024-05-27T07:02:01.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 Processor -- start 2024-05-27T07:02:01.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 -- start start 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a74640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a83030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b2770 con 0x55a828a78e70 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b42b0 con 0x55a8289e59f0 2024-05-27T07:02:01.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.744+0000 7fec60509b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b2e40 con 0x55a8288f9e40 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.745+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.745+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59160/0 (socket says 172.21.15.16:59160) 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.745+0000 7fec4b400700 1 -- 172.21.15.16:0/1042896815 learned_addr learned my addr 172.21.15.16:0/1042896815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.745+0000 7fec4aa00700 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a74640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.745+0000 7fec4a000700 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a83030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 -- 172.21.15.16:0/1042896815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 msgr2=0x55a828a74640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a74640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 -- 172.21.15.16:0/1042896815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 msgr2=0x55a828a83030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a83030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 -- 172.21.15.16:0/1042896815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a828a70240 con 0x55a828a78e70 2024-05-27T07:02:01.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec4b400700 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec440a9e70 tx=0x7fec440a9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4713289d598a44b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec37400700 1 -- 172.21.15.16:0/1042896815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440a9320 con 0x55a828a78e70 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec37400700 1 -- 172.21.15.16:0/1042896815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.746+0000 7fec37400700 1 -- 172.21.15.16:0/1042896815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440912e0 con 0x55a828a78e70 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 -- 172.21.15.16:0/1042896815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 msgr2=0x55a828a7de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec440a9e70 tx=0x7fec440a9230 comp rx=0 tx=0).stop 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 -- 172.21.15.16:0/1042896815 shutdown_connections 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a7de30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a74640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 --2- 172.21.15.16:0/1042896815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a83030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.747+0000 7fec60509b80 1 -- 172.21.15.16:0/1042896815 >> 172.21.15.16:0/1042896815 conn(0x55a828993390 msgr2=0x55a828a78720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.748+0000 7fec60509b80 1 -- 172.21.15.16:0/1042896815 shutdown_connections 2024-05-27T07:02:01.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.748+0000 7fec60509b80 1 -- 172.21.15.16:0/1042896815 wait complete. 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.748+0000 7fec60509b80 1 -- start start 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.748+0000 7fec60509b80 1 Processor -- start 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a92890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a92dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b3be0 con 0x55a828a78e70 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b3510 con 0x55a8289e59f0 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b0710 con 0x55a8288f9e40 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec60509b80 10 client.0 _ll_register_callbacks cb 0x55a828940c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:01.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec4b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59164/0 (socket says 172.21.15.16:59164) 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.749+0000 7fec4b400700 1 -- 172.21.15.16:0/3013001849 learned_addr learned my addr 172.21.15.16:0/3013001849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4aa00700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a92890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4a000700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a92dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 -- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 msgr2=0x55a828a92890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a92890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 -- 172.21.15.16:0/3013001849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 msgr2=0x55a828a92dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a92dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 -- 172.21.15.16:0/3013001849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a828afb0e0 con 0x55a828a78e70 2024-05-27T07:02:01.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec4b400700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec44007d40 tx=0x7fec440dfcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73a61ee9700c7b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:01.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec34c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.750+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44088060 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fec440a5e00 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fec440adbc0 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 10 client.15564 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a828afb0e0 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec440a9f10 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fec440a84e0 con 0x55a828a78e70 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 10 client.15564 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 20 client.15564 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828afc580 0x55a828afe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:01.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 4 client.15564 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:01.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.751+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8289a2570 con 0x55a828afc580 2024-05-27T07:02:01.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.752+0000 7fec4aa00700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828afc580 0x55a828afe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.753+0000 7fec4aa00700 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828afc580 0x55a828afe9a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbe50ff7a765a5ad server_cookie=7d192935895e1287 in_seq=0 out_seq=0 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.753+0000 7fec34c00700 10 client.15564 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec34c00700 1 -- 172.21.15.16:0/3013001849 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8289a2570 con 0x55a828afc580 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec34c00700 10 client.15564 handle_command_reply: tid=0 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 client.15564 shutdown 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 msgr2=0x55a828a93310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec44007d40 tx=0x7fec440dfcb0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 shutdown_connections 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55a828afc580 0x55a828afe9a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a828a78e70 0x55a828a93310 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8288f9e40 0x55a828a92890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 --2- 172.21.15.16:0/3013001849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8289e59f0 0x55a828a92dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec34c00700 0 client.15564 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 >> 172.21.15.16:0/3013001849 conn(0x55a828993390 msgr2=0x55a828a78720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 shutdown_connections 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 1 -- 172.21.15.16:0/3013001849 wait complete. 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.754+0000 7fec60509b80 20 client.15564 trim_cache size 0 max 16384 2024-05-27T07:02:01.748 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 msgr2=0x7fec300b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 0x7fec300b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec3800b6c0 tx=0x7fec38012890 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 msgr2=0x7fec4c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec44007d70 tx=0x7fec4400a340 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 shutdown_connections 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fec300b2b90 0x7fec300b50a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec4c110c10 0x7fec4c1a17f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c075ad0 0x7fec4c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 --2- 172.21.15.16:0/2345968141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4c076420 0x7fec4c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 >> 172.21.15.16:0/2345968141 conn(0x7fec4c100030 msgr2=0x7fec4c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 shutdown_connections 2024-05-27T07:02:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:01.755+0000 7fec51a00700 1 -- 172.21.15.16:0/2345968141 wait complete. 2024-05-27T07:02:01.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:01 smithi164 ceph-mon[35327]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:02:01.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:01 smithi069 ceph-mon[35276]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-27T07:02:02.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:02:03.285 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:03.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:03 smithi016 ceph-mon[31415]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T07:02:03.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:03 smithi016 ceph-mon[31415]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-05-27T07:02:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:03 smithi164 ceph-mon[35327]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T07:02:03.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:03 smithi164 ceph-mon[35327]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-05-27T07:02:03.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:03 smithi069 ceph-mon[35276]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T07:02:03.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:03 smithi069 ceph-mon[35276]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-05-27T07:02:05.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:05 smithi016 ceph-mon[31415]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:05.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:05 smithi164 ceph-mon[35327]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:05.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:05 smithi069 ceph-mon[35276]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.910+0000 7f2eaca00700 1 -- 172.21.15.16:0/1787423507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0003b40 con 0x7f2eb01054d0 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 -- 172.21.15.16:0/1787423507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01054d0 msgr2=0x7f2eb01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 --2- 172.21.15.16:0/1787423507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01054d0 0x7f2eb01058e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009990 tx=0x7f2ea0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 -- 172.21.15.16:0/1787423507 shutdown_connections 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 --2- 172.21.15.16:0/1787423507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb010e900 0x7f2eb0110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 --2- 172.21.15.16:0/1787423507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01054d0 0x7f2eb01058e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 --2- 172.21.15.16:0/1787423507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 0x7f2eb010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 -- 172.21.15.16:0/1787423507 >> 172.21.15.16:0/1787423507 conn(0x7f2eb00fedb0 msgr2=0x7f2eb01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 -- 172.21.15.16:0/1787423507 shutdown_connections 2024-05-27T07:02:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.911+0000 7f2eb5200700 1 -- 172.21.15.16:0/1787423507 wait complete. 2024-05-27T07:02:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 Processor -- start 2024-05-27T07:02:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 -- start start 2024-05-27T07:02:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb01054d0 0x7f2eb019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 0x7f2eb019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01142f0 con 0x7f2eb010e900 2024-05-27T07:02:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0114170 con 0x7f2eb0105e20 2024-05-27T07:02:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.913+0000 7f2eb5200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0114470 con 0x7f2eb01054d0 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.914+0000 7f2eae800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.914+0000 7f2eae800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36958/0 (socket says 172.21.15.16:36958) 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.914+0000 7f2eade00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb01054d0 0x7f2eb019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.914+0000 7f2ead400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 0x7f2eb019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.914+0000 7f2eae800700 1 -- 172.21.15.16:0/702735511 learned_addr learned my addr 172.21.15.16:0/702735511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 -- 172.21.15.16:0/702735511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb01054d0 msgr2=0x7f2eb019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb01054d0 0x7f2eb019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 -- 172.21.15.16:0/702735511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 msgr2=0x7f2eb019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 0x7f2eb019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 -- 172.21.15.16:0/702735511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb01a1df0 con 0x7f2eb010e900 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.915+0000 7f2eae800700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8007ee0 tx=0x7f2ea800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.916+0000 7f2ea6a00700 1 -- 172.21.15.16:0/702735511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004bb0 con 0x7f2eb010e900 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.916+0000 7f2ea6a00700 1 -- 172.21.15.16:0/702735511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ea8004d50 con 0x7f2eb010e900 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.916+0000 7f2ea6a00700 1 -- 172.21.15.16:0/702735511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80058b0 con 0x7f2eb010e900 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.916+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb01a20e0 con 0x7f2eb010e900 2024-05-27T07:02:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.917+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a9a00 con 0x7f2eb010e900 2024-05-27T07:02:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.922+0000 7f2ec3d6cb80 1 Processor -- start 2024-05-27T07:02:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.922+0000 7f2ea6a00700 1 -- 172.21.15.16:0/702735511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 149936+0+0 (secure 0 0 0) 0x7f2ea801a070 con 0x7f2eb010e900 2024-05-27T07:02:05.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 -- start start 2024-05-27T07:02:05.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fbdc9f0 0x55a34fc6f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a34fb9de40 con 0x55a34fc6fe70 2024-05-27T07:02:05.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a34fb8d9e0 con 0x55a34fbdc9f0 2024-05-27T07:02:05.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.923+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a34fb91a20 con 0x55a34faf0e40 2024-05-27T07:02:05.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36974/0 (socket says 172.21.15.16:36974) 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 -- 172.21.15.16:0/3974287658 learned_addr learned my addr 172.21.15.16:0/3974287658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2eade00700 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fbdc9f0 0x55a34fc6f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 -- 172.21.15.16:0/3974287658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 msgr2=0x55a34fc6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.924+0000 7f2ead400700 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc6f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ead400700 1 -- 172.21.15.16:0/3974287658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fbdc9f0 msgr2=0x55a34fc6f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ead400700 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fbdc9f0 0x55a34fc6f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ead400700 1 -- 172.21.15.16:0/3974287658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34fb56420 con 0x55a34fc6fe70 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ea6a00700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 0x7f2e940b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ea6a00700 1 -- 172.21.15.16:0/702735511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2ea80e1980 con 0x7f2eb010e900 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2ead400700 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c012810 tx=0x7f2e9c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e7495b45681fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2eade00700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 0x7f2e940b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.925+0000 7f2e9b400700 1 -- 172.21.15.16:0/3974287658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c012e30 con 0x55a34fc6fe70 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.926+0000 7f2e9b400700 1 -- 172.21.15.16:0/3974287658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c0044c0 con 0x55a34fc6fe70 2024-05-27T07:02:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.926+0000 7f2e9b400700 1 -- 172.21.15.16:0/3974287658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c005280 con 0x55a34fc6fe70 2024-05-27T07:02:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3974287658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 msgr2=0x55a34fc7c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2eade00700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 0x7f2e940b50b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00098e0 tx=0x7f2ea0005040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c012810 tx=0x7f2e9c00b860 comp rx=0 tx=0).stop 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3974287658 shutdown_connections 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7c110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fbdc9f0 0x55a34fc6f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3974287658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3974287658 >> 172.21.15.16:0/3974287658 conn(0x55a34fb8a390 msgr2=0x55a34fc6b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3974287658 shutdown_connections 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.927+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3974287658 wait complete. 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.928+0000 7f2ec3d6cb80 1 -- start start 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 Processor -- start 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fc6fe70 0x55a34fc7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a34fa93fa0 con 0x55a34fbdc9f0 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a34fb2e760 con 0x55a34faf0e40 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a34fc7cb80 con 0x55a34fc6fe70 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.929+0000 7f2ec3d6cb80 10 client.0 _ll_register_callbacks cb 0x55a34fb37c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.930+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.930+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36988/0 (socket says 172.21.15.16:36988) 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.930+0000 7f2ead400700 1 -- 172.21.15.16:0/2784025151 learned_addr learned my addr 172.21.15.16:0/2784025151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.930+0000 7f2eade00700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.930+0000 7f2eae800700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fc6fe70 0x55a34fc7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 -- 172.21.15.16:0/2784025151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fc6fe70 msgr2=0x55a34fc7a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fc6fe70 0x55a34fc7a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 -- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 msgr2=0x55a34fc796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 -- 172.21.15.16:0/2784025151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34fadd8d0 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ead400700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004980 tx=0x7f2e9c0049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee79ae7433759048 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2e8fe00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c023070 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.931+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9c0044c0 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2e9c005230 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2e9c02f050 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2ec3d6cb80 10 client.15576 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a34fb5e060 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c0044c0 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.932+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2e9c01f900 con 0x55a34fbdc9f0 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 10 client.15576 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:02:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 20 client.15576 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fc89a00 0x55a34fc8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 4 client.15576 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a34fb9fd90 con 0x55a34fc89a00 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.933+0000 7f2eade00700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fc89a00 0x55a34fc8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.935+0000 7f2e8fe00700 10 client.15576 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.935+0000 7f2eade00700 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fc89a00 0x55a34fc8be20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d41b56194d0fa96 server_cookie=75d1d609ef30bdfd in_seq=0 out_seq=0 2024-05-27T07:02:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2e8fe00700 1 -- 172.21.15.16:0/2784025151 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a34fb9fd90 con 0x55a34fc89a00 2024-05-27T07:02:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2e8fe00700 10 client.15576 handle_command_reply: tid=0 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 client.15576 shutdown 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 msgr2=0x55a34fc79c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004980 tx=0x7f2e9c0049b0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 shutdown_connections 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fc6fe70 0x55a34fc7a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fc89a00 0x55a34fc8be20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/2784025151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc79c20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2e8fe00700 0 client.15576 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 >> 172.21.15.16:0/2784025151 conn(0x55a34fb8a390 msgr2=0x55a34fc6f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 shutdown_connections 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.937+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/2784025151 wait complete. 2024-05-27T07:02:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.938+0000 7f2ec3d6cb80 20 client.15576 trim_cache size 0 max 16384 2024-05-27T07:02:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 Processor -- start 2024-05-27T07:02:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 -- start start 2024-05-27T07:02:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc73910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fc6fe70 0x55a34fc6f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b2780 con 0x55a34fbdc9f0 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b42c0 con 0x55a34fc6fe70 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.956+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b2e50 con 0x55a34faf0e40 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36990/0 (socket says 172.21.15.16:36990) 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 -- 172.21.15.16:0/4195356420 learned_addr learned my addr 172.21.15.16:0/4195356420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2eade00700 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc73910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2eae800700 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fc6fe70 0x55a34fc6f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 -- 172.21.15.16:0/4195356420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 msgr2=0x55a34fc73910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc73910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 -- 172.21.15.16:0/4195356420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fc6fe70 msgr2=0x55a34fc6f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fc6fe70 0x55a34fc6f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.957+0000 7f2ead400700 1 -- 172.21.15.16:0/4195356420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34fc67240 con 0x55a34fbdc9f0 2024-05-27T07:02:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.958+0000 7f2ead400700 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00bcc0 tx=0x7f2e9c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29715b967ca32ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.958+0000 7f2e9aa00700 1 -- 172.21.15.16:0/4195356420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c023040 con 0x55a34fbdc9f0 2024-05-27T07:02:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.958+0000 7f2e9aa00700 1 -- 172.21.15.16:0/4195356420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c016040 con 0x55a34fbdc9f0 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.958+0000 7f2e9aa00700 1 -- 172.21.15.16:0/4195356420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c005300 con 0x55a34fbdc9f0 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/4195356420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 msgr2=0x55a34fc74ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00bcc0 tx=0x7f2e9c00b860 comp rx=0 tx=0).stop 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/4195356420 shutdown_connections 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34fc6fe70 0x55a34fc6f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34faf0e40 0x55a34fc73910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/4195356420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fbdc9f0 0x55a34fc74ec0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/4195356420 >> 172.21.15.16:0/4195356420 conn(0x55a34fb8a390 msgr2=0x55a34fc796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.959+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/4195356420 shutdown_connections 2024-05-27T07:02:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.960+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/4195356420 wait complete. 2024-05-27T07:02:05.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.960+0000 7f2ec3d6cb80 1 -- start start 2024-05-27T07:02:05.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.960+0000 7f2ec3d6cb80 1 Processor -- start 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc7cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 0x55a34fc7d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b3bf0 con 0x55a34fc6fe70 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b3520 con 0x55a34faf0e40 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01b0720 con 0x55a34fbdc9f0 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2ec3d6cb80 10 client.0 _ll_register_callbacks cb 0x55a34fb37c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.961+0000 7f2eae800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36994/0 (socket says 172.21.15.16:36994) 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 -- 172.21.15.16:0/3975635981 learned_addr learned my addr 172.21.15.16:0/3975635981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eade00700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc7cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2ead400700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 0x55a34fc7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 -- 172.21.15.16:0/3975635981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 msgr2=0x55a34fc7d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 0x55a34fc7d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 -- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 msgr2=0x55a34fc7cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc7cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 -- 172.21.15.16:0/3975635981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34fcf2280 con 0x55a34fc6fe70 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2ead400700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 0x55a34fc7d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eade00700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc7cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2eae800700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80debf0 tx=0x7f2ea8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b620857d49d383e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.962+0000 7f2e8fe00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80a8d90 con 0x55a34fc6fe70 2024-05-27T07:02:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2ea80b1270 con 0x55a34fc6fe70 2024-05-27T07:02:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9c016040 con 0x55a34fc6fe70 2024-05-27T07:02:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ea80a8690 con 0x55a34fc6fe70 2024-05-27T07:02:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2ec3d6cb80 10 client.15582 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a34fcf3470 con 0x55a34fc6fe70 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.963+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c016040 con 0x55a34fc6fe70 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2ea800db20 con 0x55a34fc6fe70 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 10 client.15582 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 20 client.15582 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fcf3b00 0x55a34fcf5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 4 client.15582 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.964+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a34fb99570 con 0x55a34fcf3b00 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.965+0000 7f2eade00700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fcf3b00 0x55a34fcf5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2e8fe00700 10 client.15582 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2eade00700 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fcf3b00 0x55a34fcf5f20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3850baa5970120f server_cookie=1dbb5eccf9acae68 in_seq=0 out_seq=0 2024-05-27T07:02:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2e8fe00700 1 -- 172.21.15.16:0/3975635981 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a34fb99570 con 0x55a34fcf3b00 2024-05-27T07:02:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2e8fe00700 10 client.15582 handle_command_reply: tid=0 2024-05-27T07:02:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2ec3d6cb80 1 client.15582 shutdown 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 msgr2=0x55a34fc7d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.966+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80debf0 tx=0x7f2ea8005230 comp rx=0 tx=0).stop 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 shutdown_connections 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a34fc6fe70 0x55a34fc7d850 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a34fcf3b00 0x55a34fcf5f20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a34faf0e40 0x55a34fc7cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 --2- 172.21.15.16:0/3975635981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a34fbdc9f0 0x55a34fc7d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2e8fe00700 0 client.15582 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 >> 172.21.15.16:0/3975635981 conn(0x55a34fb8a390 msgr2=0x55a34fc78a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 shutdown_connections 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 1 -- 172.21.15.16:0/3975635981 wait complete. 2024-05-27T07:02:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.967+0000 7f2ec3d6cb80 20 client.15582 trim_cache size 0 max 16384 2024-05-27T07:02:05.961 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 msgr2=0x7f2e940b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 0x7f2e940b50b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00098e0 tx=0x7f2ea0005040 comp rx=0 tx=0).stop 2024-05-27T07:02:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 msgr2=0x7f2eb01a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8007ee0 tx=0x7f2ea800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 shutdown_connections 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2e940b2ba0 0x7f2e940b50b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010e900 0x7f2eb01a1750 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2eb01054d0 0x7f2eb019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 --2- 172.21.15.16:0/702735511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eb0105e20 0x7f2eb019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 >> 172.21.15.16:0/702735511 conn(0x7f2eb00fedb0 msgr2=0x7f2eb00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 shutdown_connections 2024-05-27T07:02:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:05.968+0000 7f2eb5200700 1 -- 172.21.15.16:0/702735511 wait complete. 2024-05-27T07:02:06.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:02:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:07.730 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:07 smithi016 ceph-mon[31415]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:07.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:07 smithi164 ceph-mon[35327]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:07.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:07 smithi069 ceph-mon[35276]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:09.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:09 smithi016 ceph-mon[31415]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T07:02:09.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:09 smithi164 ceph-mon[35327]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T07:02:09.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:09 smithi069 ceph-mon[35276]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T07:02:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 -- 172.21.15.16:0/1897893958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568115170 msgr2=0x7fe568117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 --2- 172.21.15.16:0/1897893958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568115170 0x7fe568117590 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0a0 tx=0x7fe56000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 -- 172.21.15.16:0/1897893958 shutdown_connections 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 --2- 172.21.15.16:0/1897893958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568115170 0x7fe568117590 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 --2- 172.21.15.16:0/1897893958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568075b50 0x7fe568075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 --2- 172.21.15.16:0/1897893958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe568114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 -- 172.21.15.16:0/1897893958 >> 172.21.15.16:0/1897893958 conn(0x7fe568079d00 msgr2=0x7fe56807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 -- 172.21.15.16:0/1897893958 shutdown_connections 2024-05-27T07:02:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.896+0000 7fe567200700 1 -- 172.21.15.16:0/1897893958 wait complete. 2024-05-27T07:02:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.897+0000 7fe567200700 1 Processor -- start 2024-05-27T07:02:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.897+0000 7fe567200700 1 -- start start 2024-05-27T07:02:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe5681a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568115170 0x7fe5681a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811a780 con 0x7fe568075b50 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811a600 con 0x7fe568115170 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe567200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811a900 con 0x7fe5680764a0 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37004/0 (socket says 172.21.15.16:37004) 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.898+0000 7fe565e00700 1 -- 172.21.15.16:0/2823808543 learned_addr learned my addr 172.21.15.16:0/2823808543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565400700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe5681a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe566800700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568115170 0x7fe5681a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 -- 172.21.15.16:0/2823808543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 msgr2=0x7fe5681a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe5681a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 -- 172.21.15.16:0/2823808543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568115170 msgr2=0x7fe5681a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568115170 0x7fe5681a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 -- 172.21.15.16:0/2823808543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681a5fe0 con 0x7fe568075b50 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565400700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe5681a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.899+0000 7fe565e00700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d950 tx=0x7fe55800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.900+0000 7fe55ea00700 1 -- 172.21.15.16:0/2823808543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558016070 con 0x7fe568075b50 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.900+0000 7fe55ea00700 1 -- 172.21.15.16:0/2823808543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe558010460 con 0x7fe568075b50 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.900+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681a62d0 con 0x7fe568075b50 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.900+0000 7fe55ea00700 1 -- 172.21.15.16:0/2823808543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55800f620 con 0x7fe568075b50 2024-05-27T07:02:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.900+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681adbf0 con 0x7fe568075b50 2024-05-27T07:02:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.904+0000 7fe55ea00700 1 -- 172.21.15.16:0/2823808543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 149936+0+0 (secure 0 0 0) 0x7fe558019070 con 0x7fe568075b50 2024-05-27T07:02:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 Processor -- start 2024-05-27T07:02:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 -- start start 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49bc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49bcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cb48e9e40 con 0x559cb483ce40 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559cb48d99e0 con 0x559cb49289f0 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.905+0000 7fe57b96db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559cb48dda20 con 0x559cb49c62c0 2024-05-27T07:02:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49bcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49bcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37016/0 (socket says 172.21.15.16:37016) 2024-05-27T07:02:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe565e00700 1 -- 172.21.15.16:0/2467535319 learned_addr learned my addr 172.21.15.16:0/2467535319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe55ea00700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 0x7fe54c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe565400700 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe566800700 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.906+0000 7fe566800700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 0x7fe54c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe55ea00700 1 -- 172.21.15.16:0/2823808543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe5580e2640 con 0x7fe568075b50 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 -- 172.21.15.16:0/2467535319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 msgr2=0x559cb49bc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49bc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 -- 172.21.15.16:0/2467535319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 msgr2=0x559cb49bcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49bcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 -- 172.21.15.16:0/2467535319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb48a2420 con 0x559cb49c62c0 2024-05-27T07:02:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.907+0000 7fe565400700 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c86e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe55400dc20 tx=0x7fe55400df30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9004a1b0004fdf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.908+0000 7fe566800700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 0x7fe54c0b50b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x559cb49b8a90 tx=0x7fe5600099e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.908+0000 7fe553400700 1 -- 172.21.15.16:0/2467535319 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400fb10 con 0x559cb49c62c0 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.908+0000 7fe553400700 1 -- 172.21.15.16:0/2467535319 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55400fcb0 con 0x559cb49c62c0 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.908+0000 7fe553400700 1 -- 172.21.15.16:0/2467535319 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55401b670 con 0x559cb49c62c0 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 -- 172.21.15.16:0/2467535319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 msgr2=0x559cb49c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c86e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe55400dc20 tx=0x7fe55400df30 comp rx=0 tx=0).stop 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 -- 172.21.15.16:0/2467535319 shutdown_connections 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c86e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49bc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2467535319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49bcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 -- 172.21.15.16:0/2467535319 >> 172.21.15.16:0/2467535319 conn(0x559cb48d6390 msgr2=0x559cb49b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 -- 172.21.15.16:0/2467535319 shutdown_connections 2024-05-27T07:02:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.909+0000 7fe57b96db80 1 -- 172.21.15.16:0/2467535319 wait complete. 2024-05-27T07:02:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.910+0000 7fe57b96db80 1 -- start start 2024-05-27T07:02:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 Processor -- start 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cb47dffa0 con 0x559cb483ce40 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559cb487a760 con 0x559cb49289f0 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.911+0000 7fe57b96db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559cb49c8e00 con 0x559cb49c62c0 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe57b96db80 10 client.0 _ll_register_callbacks cb 0x559cb4883c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe565e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37030/0 (socket says 172.21.15.16:37030) 2024-05-27T07:02:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe565e00700 1 -- 172.21.15.16:0/3068669425 learned_addr learned my addr 172.21.15.16:0/3068669425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe565400700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.912+0000 7fe566800700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 -- 172.21.15.16:0/3068669425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 msgr2=0x559cb49c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 -- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 msgr2=0x559cb49c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 -- 172.21.15.16:0/3068669425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb48298d0 con 0x559cb483ce40 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe565e00700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe5580df730 tx=0x7fe558010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c6a7661d16e4ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe547e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580e2070 con 0x559cb483ce40 2024-05-27T07:02:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.913+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe5580e2210 con 0x559cb483ce40 2024-05-27T07:02:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55400fcb0 con 0x559cb483ce40 2024-05-27T07:02:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe558020040 con 0x559cb483ce40 2024-05-27T07:02:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe57b96db80 10 client.15594 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559cb48aa060 con 0x559cb483ce40 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55400fcb0 con 0x559cb483ce40 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe558023060 con 0x559cb483ce40 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe57b96db80 10 client.15594 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.914+0000 7fe57b96db80 10 client.15594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.915+0000 7fe57b96db80 20 client.15594 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.915+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb49d5fc0 0x559cb49d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.915+0000 7fe57b96db80 4 client.15594 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.915+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cb48ebd90 con 0x559cb49d5fc0 2024-05-27T07:02:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.915+0000 7fe565400700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb49d5fc0 0x559cb49d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.917+0000 7fe565400700 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb49d5fc0 0x559cb49d83e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=452120dcef65e464 server_cookie=680df27c371f3ed2 in_seq=0 out_seq=0 2024-05-27T07:02:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.917+0000 7fe547e00700 10 client.15594 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe547e00700 1 -- 172.21.15.16:0/3068669425 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559cb48ebd90 con 0x559cb49d5fc0 2024-05-27T07:02:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe547e00700 10 client.15594 handle_command_reply: tid=0 2024-05-27T07:02:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 client.15594 shutdown 2024-05-27T07:02:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 msgr2=0x559cb49c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe5580df730 tx=0x7fe558010c10 comp rx=0 tx=0).stop 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 shutdown_connections 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb49d5fc0 0x559cb49d83e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb483ce40 0x559cb49c3250 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 --2- 172.21.15.16:0/3068669425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb49289f0 0x559cb49c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 >> 172.21.15.16:0/3068669425 conn(0x559cb48d6390 msgr2=0x559cb49b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe547e00700 0 client.15594 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 shutdown_connections 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.919+0000 7fe57b96db80 1 -- 172.21.15.16:0/3068669425 wait complete. 2024-05-27T07:02:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.920+0000 7fe57b96db80 20 client.15594 trim_cache size 0 max 16384 2024-05-27T07:02:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.952+0000 7fe57b96db80 1 Processor -- start 2024-05-27T07:02:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.952+0000 7fe57b96db80 1 -- start start 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b6900 con 0x559cb49289f0 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b8440 con 0x559cb483ce40 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe57b96db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b6fd0 con 0x559cb49c62c0 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe565400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe565400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37042/0 (socket says 172.21.15.16:37042) 2024-05-27T07:02:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe565400700 1 -- 172.21.15.16:0/2739145052 learned_addr learned my addr 172.21.15.16:0/2739145052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.953+0000 7fe565e00700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe566800700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 -- 172.21.15.16:0/2739145052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 msgr2=0x559cb49db400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49db400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 -- 172.21.15.16:0/2739145052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 msgr2=0x559cb49d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 -- 172.21.15.16:0/2739145052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb49b7c50 con 0x559cb49289f0 2024-05-27T07:02:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565e00700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe565400700 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe55400dc20 tx=0x7fe554011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416f69006cf0a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.954+0000 7fe552a00700 1 -- 172.21.15.16:0/2739145052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400fc20 con 0x559cb49289f0 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.955+0000 7fe552a00700 1 -- 172.21.15.16:0/2739145052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5540044d0 con 0x559cb49289f0 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.955+0000 7fe552a00700 1 -- 172.21.15.16:0/2739145052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004710 con 0x559cb49289f0 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- 172.21.15.16:0/2739145052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 msgr2=0x559cb49bb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe55400dc20 tx=0x7fe554011d90 comp rx=0 tx=0).stop 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- 172.21.15.16:0/2739145052 shutdown_connections 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49c62c0 0x559cb49db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 --2- 172.21.15.16:0/2739145052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49289f0 0x559cb49bb830 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- 172.21.15.16:0/2739145052 >> 172.21.15.16:0/2739145052 conn(0x559cb48d6390 msgr2=0x559cb49bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- 172.21.15.16:0/2739145052 shutdown_connections 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- 172.21.15.16:0/2739145052 wait complete. 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.956+0000 7fe57b96db80 1 -- start start 2024-05-27T07:02:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 Processor -- start 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49289f0 0x559cb49d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b7d70 con 0x559cb49c62c0 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b76a0 con 0x559cb483ce40 2024-05-27T07:02:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681b48a0 con 0x559cb49289f0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.957+0000 7fe57b96db80 10 client.0 _ll_register_callbacks cb 0x559cb4883c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37052/0 (socket says 172.21.15.16:37052) 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 -- 172.21.15.16:0/4293545287 learned_addr learned my addr 172.21.15.16:0/4293545287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe565400700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49289f0 0x559cb49d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe565e00700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 -- 172.21.15.16:0/4293545287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49289f0 msgr2=0x559cb49d8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49289f0 0x559cb49d8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 -- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 msgr2=0x559cb49d8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe566800700 1 -- 172.21.15.16:0/4293545287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb49c4040 con 0x559cb49c62c0 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.958+0000 7fe565e00700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe566800700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0a0 tx=0x7fe56000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26db2d34e37d99d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe547e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560030030 con 0x559cb49c62c0 2024-05-27T07:02:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5540044d0 con 0x559cb49c62c0 2024-05-27T07:02:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe560026040 con 0x559cb49c62c0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe56002c450 con 0x559cb49c62c0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe57b96db80 10 client.15600 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.959+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559cb4a3fbe0 con 0x559cb49c62c0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5540044d0 con 0x559cb49c62c0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe560011070 con 0x559cb49c62c0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 10 client.15600 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 20 client.15600 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb4a402a0 0x559cb4a426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 4 client.15600 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.960+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cb48e5570 con 0x559cb4a402a0 2024-05-27T07:02:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.961+0000 7fe565e00700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb4a402a0 0x559cb4a426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe565e00700 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb4a402a0 0x559cb4a426c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97b05448141b9bf3 server_cookie=29ccd18c6470264a in_seq=0 out_seq=0 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe547e00700 10 client.15600 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe547e00700 1 -- 172.21.15.16:0/4293545287 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cb48e5570 con 0x559cb4a402a0 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe547e00700 10 client.15600 handle_command_reply: tid=0 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe57b96db80 1 client.15600 shutdown 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 msgr2=0x559cb4a3e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.962+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0a0 tx=0x7fe56000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 shutdown_connections 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x559cb4a402a0 0x559cb4a426c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cb49c62c0 0x559cb4a3e820 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559cb483ce40 0x559cb49d8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 --2- 172.21.15.16:0/4293545287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559cb49289f0 0x559cb49d8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 >> 172.21.15.16:0/4293545287 conn(0x559cb48d6390 msgr2=0x559cb49c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe547e00700 0 client.15600 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 shutdown_connections 2024-05-27T07:02:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 1 -- 172.21.15.16:0/4293545287 wait complete. 2024-05-27T07:02:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.963+0000 7fe57b96db80 20 client.15600 trim_cache size 0 max 16384 2024-05-27T07:02:09.958 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 msgr2=0x7fe54c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 0x7fe54c0b50b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x559cb49b8a90 tx=0x7fe5600099e0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 msgr2=0x7fe5681a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d950 tx=0x7fe55800dd10 comp rx=0 tx=0).stop 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 shutdown_connections 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe54c0b2ba0 0x7fe54c0b50b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe568115170 0x7fe5681a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568075b50 0x7fe5681a1110 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 --2- 172.21.15.16:0/2823808543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5680764a0 0x7fe5681a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 >> 172.21.15.16:0/2823808543 conn(0x7fe568079d00 msgr2=0x7fe568100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 shutdown_connections 2024-05-27T07:02:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:09.964+0000 7fe567200700 1 -- 172.21.15.16:0/2823808543 wait complete. 2024-05-27T07:02:11.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:02:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:11.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:11 smithi164 ceph-mon[35327]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-27T07:02:11.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:11 smithi016 ceph-mon[31415]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-27T07:02:11.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:11 smithi069 ceph-mon[35276]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-27T07:02:13.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:13 smithi164 ceph-mon[35327]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:13.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:13 smithi016 ceph-mon[31415]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:13.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:13 smithi069 ceph-mon[35276]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 -- 172.21.15.16:0/115536249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105c50 msgr2=0x7f5e0810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 --2- 172.21.15.16:0/115536249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105c50 0x7f5e0810e1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009990 tx=0x7f5df4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 -- 172.21.15.16:0/115536249 shutdown_connections 2024-05-27T07:02:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 --2- 172.21.15.16:0/115536249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e08110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 --2- 172.21.15.16:0/115536249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08105300 0x7f5e08105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 --2- 172.21.15.16:0/115536249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105c50 0x7f5e0810e1a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 -- 172.21.15.16:0/115536249 >> 172.21.15.16:0/115536249 conn(0x7f5e08078af0 msgr2=0x7f5e08078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.105+0000 7f5e0d200700 1 -- 172.21.15.16:0/115536249 shutdown_connections 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.106+0000 7f5e0d200700 1 -- 172.21.15.16:0/115536249 wait complete. 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.107+0000 7f5e0d200700 1 Processor -- start 2024-05-27T07:02:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.107+0000 7f5e0d200700 1 -- start start 2024-05-27T07:02:14.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08105c50 0x7f5e0819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e081a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08114160 con 0x7f5e08105300 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08113fe0 con 0x7f5e0810e730 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e0d200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081142e0 con 0x7f5e08105c50 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e07400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e07400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-05-27T07:02:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.108+0000 7f5e07400700 1 -- 172.21.15.16:0/1826225864 learned_addr learned my addr 172.21.15.16:0/1826225864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:14.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e081a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 -- 172.21.15.16:0/1826225864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08105c50 msgr2=0x7f5e0819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08105c50 0x7f5e0819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 -- 172.21.15.16:0/1826225864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 msgr2=0x7f5e081a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e081a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 -- 172.21.15.16:0/1826225864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e081a1c80 con 0x7f5e08105300 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e081a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.109+0000 7f5e07400700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5df800d950 tx=0x7f5df800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.110+0000 7f5dffe00700 1 -- 172.21.15.16:0/1826225864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8016070 con 0x7f5e08105300 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.110+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e081a1f70 con 0x7f5e08105300 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.110+0000 7f5dffe00700 1 -- 172.21.15.16:0/1826225864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df8010460 con 0x7f5e08105300 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.110+0000 7f5dffe00700 1 -- 172.21.15.16:0/1826225864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df800f620 con 0x7f5e08105300 2024-05-27T07:02:14.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.110+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081a9890 con 0x7f5e08105300 2024-05-27T07:02:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.114+0000 7f5dffe00700 1 -- 172.21.15.16:0/1826225864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 149936+0+0 (secure 0 0 0) 0x7f5df8019070 con 0x7f5e08105300 2024-05-27T07:02:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.116+0000 7f5e1d34fb80 1 Processor -- start 2024-05-27T07:02:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.116+0000 7f5e1d34fb80 1 -- start start 2024-05-27T07:02:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5dffe00700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 0x7f5dec0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d410 0x560453a57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560453a4d820 0x560453a5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56045397be40 con 0x560453a4d410 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56045396b9e0 con 0x560453a4d820 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56045396fa20 con 0x5604538cee40 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5dffe00700 1 -- 172.21.15.16:0/1826225864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5df80e2940 con 0x7f5e08105300 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 0x7f5dec0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e07400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e07400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e07400700 1 -- 172.21.15.16:0/994837884 learned_addr learned my addr 172.21.15.16:0/994837884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.117+0000 7f5e06a00700 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d410 0x560453a57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07e00700 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560453a4d820 0x560453a5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07400700 1 -- 172.21.15.16:0/994837884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560453a4d820 msgr2=0x560453a5a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07400700 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560453a4d820 0x560453a5a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07400700 1 -- 172.21.15.16:0/994837884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d410 msgr2=0x560453a57ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07400700 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d410 0x560453a57ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07400700 1 -- 172.21.15.16:0/994837884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560453934420 con 0x5604538cee40 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.118+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 0x7f5dec0b50b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009790 tx=0x7f5e000079a0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.119+0000 7f5e07400700 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5df80dfbb0 tx=0x7f5df80a8e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5776a8bd0334ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.119+0000 7f5dfcc00700 1 -- 172.21.15.16:0/994837884 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80a9570 con 0x5604538cee40 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.119+0000 7f5dfcc00700 1 -- 172.21.15.16:0/994837884 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df80ad920 con 0x5604538cee40 2024-05-27T07:02:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.119+0000 7f5dfcc00700 1 -- 172.21.15.16:0/994837884 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80d60a0 con 0x5604538cee40 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.120+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/994837884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 msgr2=0x560453a4ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.120+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5df80dfbb0 tx=0x7f5df80a8e10 comp rx=0 tx=0).stop 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/994837884 shutdown_connections 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560453a4d820 0x560453a5a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604538cee40 0x560453a4ced0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/994837884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d410 0x560453a57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/994837884 >> 172.21.15.16:0/994837884 conn(0x56045392efc0 msgr2=0x560453a48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/994837884 shutdown_connections 2024-05-27T07:02:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.121+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/994837884 wait complete. 2024-05-27T07:02:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.122+0000 7f5e1d34fb80 1 -- start start 2024-05-27T07:02:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.122+0000 7f5e1d34fb80 1 Processor -- start 2024-05-27T07:02:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560453871fa0 con 0x560453a4d820 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56045390c760 con 0x5604538cee40 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560453a5a990 con 0x560453a4d410 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e1d34fb80 10 client.0 _ll_register_callbacks cb 0x560453915c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41802/0 (socket says 172.21.15.16:41802) 2024-05-27T07:02:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e07e00700 1 -- 172.21.15.16:0/774027095 learned_addr learned my addr 172.21.15.16:0/774027095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e06a00700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.123+0000 7f5e07400700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 -- 172.21.15.16:0/774027095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 msgr2=0x560453a57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 -- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 msgr2=0x560453a57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a57630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 -- 172.21.15.16:0/774027095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604538bb8d0 con 0x560453a4d820 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e07e00700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009370 tx=0x7f5e00004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558c6a8195186c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5df1600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00028060 con 0x560453a4d820 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5e00003c00 con 0x560453a4d820 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.124+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df80ad920 con 0x560453a4d820 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.125+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e0002f050 con 0x560453a4d820 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.125+0000 7f5e1d34fb80 10 client.15612 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.125+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56045393c060 con 0x560453a4d820 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.125+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df80ad920 con 0x560453a4d820 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.125+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5e00018070 con 0x560453a4d820 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 10 client.15612 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 20 client.15612 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453a679d0 0x560453a69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 4 client.15612 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56045397dd90 con 0x560453a679d0 2024-05-27T07:02:14.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.126+0000 7f5e07400700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453a679d0 0x560453a69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.127+0000 7f5e07400700 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453a679d0 0x560453a69df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f125ebde1cafe72 server_cookie=8be7d5b51361c05f in_seq=0 out_seq=0 2024-05-27T07:02:14.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.127+0000 7f5df1600700 10 client.15612 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:14.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.129+0000 7f5df1600700 1 -- 172.21.15.16:0/774027095 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56045397dd90 con 0x560453a679d0 2024-05-27T07:02:14.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.129+0000 7f5df1600700 10 client.15612 handle_command_reply: tid=0 2024-05-27T07:02:14.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.129+0000 7f5e1d34fb80 1 client.15612 shutdown 2024-05-27T07:02:14.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 msgr2=0x560453a580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009370 tx=0x7f5e00004b40 comp rx=0 tx=0).stop 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 shutdown_connections 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453a679d0 0x560453a69df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a580b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/774027095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 >> 172.21.15.16:0/774027095 conn(0x56045392efc0 msgr2=0x560453a47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5df1600700 0 client.15612 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 shutdown_connections 2024-05-27T07:02:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.130+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/774027095 wait complete. 2024-05-27T07:02:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.131+0000 7f5e1d34fb80 20 client.15612 trim_cache size 0 max 16384 2024-05-27T07:02:14.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 Processor -- start 2024-05-27T07:02:14.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 -- start start 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b26d0 con 0x560453a4d820 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b4210 con 0x5604538cee40 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.151+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b2da0 con 0x560453a4d410 2024-05-27T07:02:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41804/0 (socket says 172.21.15.16:41804) 2024-05-27T07:02:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 -- 172.21.15.16:0/367393142 learned_addr learned my addr 172.21.15.16:0/367393142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07400700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e06a00700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 -- 172.21.15.16:0/367393142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 msgr2=0x560453a59ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a59ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 -- 172.21.15.16:0/367393142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 msgr2=0x560453a52e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a52e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07e00700 1 -- 172.21.15.16:0/367393142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560453a431d0 con 0x560453a4d820 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.152+0000 7f5e07400700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.153+0000 7f5e07e00700 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e00007e50 tx=0x7f5e00009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c377dd40d6f4078 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.153+0000 7f5df3e00700 1 -- 172.21.15.16:0/367393142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00028070 con 0x560453a4d820 2024-05-27T07:02:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.153+0000 7f5df3e00700 1 -- 172.21.15.16:0/367393142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0001f040 con 0x560453a4d820 2024-05-27T07:02:14.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.153+0000 7f5df3e00700 1 -- 172.21.15.16:0/367393142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00012040 con 0x560453a4d820 2024-05-27T07:02:14.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/367393142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 msgr2=0x560453a6f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e00007e50 tx=0x7f5e00009d10 comp rx=0 tx=0).stop 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/367393142 shutdown_connections 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a6f570 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453a52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/367393142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453a59ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.154+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/367393142 >> 172.21.15.16:0/367393142 conn(0x56045392efc0 msgr2=0x560453a55f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.155+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/367393142 shutdown_connections 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.155+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/367393142 wait complete. 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.155+0000 7f5e1d34fb80 1 -- start start 2024-05-27T07:02:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.155+0000 7f5e1d34fb80 1 Processor -- start 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453aca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453acad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b3b40 con 0x560453a4d820 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b3470 con 0x5604538cee40 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b0670 con 0x560453a4d410 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.156+0000 7f5e1d34fb80 10 client.0 _ll_register_callbacks cb 0x560453915c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41814/0 (socket says 172.21.15.16:41814) 2024-05-27T07:02:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 -- 172.21.15.16:0/1358729441 learned_addr learned my addr 172.21.15.16:0/1358729441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07400700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453aca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e06a00700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453acad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 -- 172.21.15.16:0/1358729441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 msgr2=0x560453acad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453acad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 -- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 msgr2=0x560453aca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453aca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.157+0000 7f5e07e00700 1 -- 172.21.15.16:0/1358729441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560453a58420 con 0x560453a4d820 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5e07e00700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001b040 tx=0x7f5e00005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6583ab5738bebc85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5df1600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00028070 con 0x560453a4d820 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5e00022b40 con 0x560453a4d820 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0001f040 con 0x560453a4d820 2024-05-27T07:02:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e00012040 con 0x560453a4d820 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5e1d34fb80 10 client.15618 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.158+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560453a4a350 con 0x560453a4d820 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e0001f040 con 0x560453a4d820 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5e00032070 con 0x560453a4d820 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 20 client.15618 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453ad20e0 0x560453ad4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 4 client.15618 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.159+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560453977570 con 0x560453ad20e0 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.160+0000 7f5e07400700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453ad20e0 0x560453ad4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.161+0000 7f5e07400700 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453ad20e0 0x560453ad4500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e965df4898c4b52 server_cookie=2a581bdf53bbda31 in_seq=0 out_seq=0 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.161+0000 7f5df1600700 10 client.15618 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.161+0000 7f5df1600700 1 -- 172.21.15.16:0/1358729441 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560453977570 con 0x560453ad20e0 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.161+0000 7f5df1600700 10 client.15618 handle_command_reply: tid=0 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.161+0000 7f5e1d34fb80 1 client.15618 shutdown 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 msgr2=0x560453a5b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e0001b040 tx=0x7f5e00005e90 comp rx=0 tx=0).stop 2024-05-27T07:02:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 shutdown_connections 2024-05-27T07:02:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560453ad20e0 0x560453ad4500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560453a4d820 0x560453a5b1c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5604538cee40 0x560453aca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 --2- 172.21.15.16:0/1358729441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560453a4d410 0x560453acad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 >> 172.21.15.16:0/1358729441 conn(0x56045392efc0 msgr2=0x560453a57410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5df1600700 0 client.15618 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 shutdown_connections 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 1 -- 172.21.15.16:0/1358729441 wait complete. 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.162+0000 7f5e1d34fb80 20 client.15618 trim_cache size 0 max 16384 2024-05-27T07:02:14.159 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.163+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 msgr2=0x7f5dec0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.163+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 0x7f5dec0b50b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009790 tx=0x7f5e000079a0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.163+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 msgr2=0x7f5e0819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.163+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5df800d950 tx=0x7f5df800dd10 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 shutdown_connections 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5dec0b2ba0 0x7f5dec0b50b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e0810e730 0x7f5e081a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e08105300 0x7f5e0819cdb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 --2- 172.21.15.16:0/1826225864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08105c50 0x7f5e0819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 >> 172.21.15.16:0/1826225864 conn(0x7f5e08078af0 msgr2=0x7f5e080ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 shutdown_connections 2024-05-27T07:02:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:14.164+0000 7f5e0d200700 1 -- 172.21.15.16:0/1826225864 wait complete. 2024-05-27T07:02:14.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:14 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:02:14.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:14 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:02:14.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:14 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:02:15.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:02:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:15.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:15 smithi016 ceph-mon[31415]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:15.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:15 smithi164 ceph-mon[35327]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:15.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:15 smithi069 ceph-mon[35276]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T07:02:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:16 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:02:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:16 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:02:16.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:16 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:02:16.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:16 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:02:16.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:16 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:02:16.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:16 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:02:16.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:16 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:02:16.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:16 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:02:16.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:16 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:02:17.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:17 smithi164 ceph-mon[35327]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:17.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:17 smithi016 ceph-mon[31415]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:17.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:17 smithi069 ceph-mon[35276]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T07:02:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:19 smithi164 ceph-mon[35327]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T07:02:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:19 smithi016 ceph-mon[31415]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T07:02:19.871 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:19 smithi069 ceph-mon[35276]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T07:02:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.089+0000 7ff12ea00700 1 -- 172.21.15.16:0/1154523383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1281034b0 msgr2=0x7ff1281038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.089+0000 7ff12ea00700 1 --2- 172.21.15.16:0/1154523383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1281034b0 0x7ff1281038c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff118009a40 tx=0x7ff118009d50 comp rx=0 tx=0).stop 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 -- 172.21.15.16:0/1154523383 shutdown_connections 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 --2- 172.21.15.16:0/1154523383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128108730 0x7ff128110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 --2- 172.21.15.16:0/1154523383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1281034b0 0x7ff1281038c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 --2- 172.21.15.16:0/1154523383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff128107de0 0x7ff1281081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 -- 172.21.15.16:0/1154523383 >> 172.21.15.16:0/1154523383 conn(0x7ff1280fedb0 msgr2=0x7ff1281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 -- 172.21.15.16:0/1154523383 shutdown_connections 2024-05-27T07:02:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.090+0000 7ff12ea00700 1 -- 172.21.15.16:0/1154523383 wait complete. 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.092+0000 7ff12ea00700 1 Processor -- start 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.092+0000 7ff12ea00700 1 -- start start 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1281034b0 0x7ff12819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128107de0 0x7ff12819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff128112170 con 0x7ff128108730 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111ff0 con 0x7ff1281034b0 2024-05-27T07:02:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281122f0 con 0x7ff128107de0 2024-05-27T07:02:21.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41824/0 (socket says 172.21.15.16:41824) 2024-05-27T07:02:21.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff12ca00700 1 -- 172.21.15.16:0/2228678771 learned_addr learned my addr 172.21.15.16:0/2228678771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff127e00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1281034b0 0x7ff12819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.093+0000 7ff127400700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128107de0 0x7ff12819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 -- 172.21.15.16:0/2228678771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128107de0 msgr2=0x7ff12819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128107de0 0x7ff12819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 -- 172.21.15.16:0/2228678771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1281034b0 msgr2=0x7ff12819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1281034b0 0x7ff12819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 -- 172.21.15.16:0/2228678771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128198fc0 con 0x7ff128108730 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.094+0000 7ff12ca00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff120007ee0 tx=0x7ff12000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.095+0000 7ff124c00700 1 -- 172.21.15.16:0/2228678771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004bb0 con 0x7ff128108730 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.095+0000 7ff124c00700 1 -- 172.21.15.16:0/2228678771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff120004d50 con 0x7ff128108730 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.095+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1281992b0 con 0x7ff128108730 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.095+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281a5580 con 0x7ff128108730 2024-05-27T07:02:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.095+0000 7ff124c00700 1 -- 172.21.15.16:0/2228678771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200058b0 con 0x7ff128108730 2024-05-27T07:02:21.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.101+0000 7ff124c00700 1 -- 172.21.15.16:0/2228678771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 149936+0+0 (secure 0 0 0) 0x7ff12001a020 con 0x7ff128108730 2024-05-27T07:02:21.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.104+0000 7ff124c00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 0x7ff10c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.104+0000 7ff124c00700 1 -- 172.21.15.16:0/2228678771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff1200e2130 con 0x7ff128108730 2024-05-27T07:02:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.104+0000 7ff13d52bb80 1 Processor -- start 2024-05-27T07:02:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff127e00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 0x7ff10c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 -- start start 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8ff6e40 con 0x55dbf8f49e40 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8fe69e0 con 0x55dbf90d74e0 2024-05-27T07:02:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.105+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8feaa20 con 0x55dbf90d15c0 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.106+0000 7ff127400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.106+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.106+0000 7ff127400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53442/0 (socket says 172.21.15.16:53442) 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.106+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33382/0 (socket says 172.21.15.16:33382) 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.106+0000 7ff127400700 1 -- 172.21.15.16:0/136484371 learned_addr learned my addr 172.21.15.16:0/136484371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127400700 1 -- 172.21.15.16:0/136484371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 msgr2=0x55dbf90d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127400700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127400700 1 -- 172.21.15.16:0/136484371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 msgr2=0x55dbf90d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127e00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 0x7ff10c0b51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff11801a040 tx=0x7ff118009920 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127e00700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127400700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff127400700 1 -- 172.21.15.16:0/136484371 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf8faf420 con 0x55dbf90d74e0 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.107+0000 7ff12ca00700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:02:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.108+0000 7ff127e00700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.108+0000 7ff127400700 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff11400bc90 tx=0x7ff11400b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578fe459f173706e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.109+0000 7ff11d600700 1 -- 172.21.15.16:0/136484371 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400acd0 con 0x55dbf90d74e0 2024-05-27T07:02:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.109+0000 7ff11d600700 1 -- 172.21.15.16:0/136484371 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1140044c0 con 0x55dbf90d74e0 2024-05-27T07:02:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.109+0000 7ff11d600700 1 -- 172.21.15.16:0/136484371 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114015400 con 0x55dbf90d74e0 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.110+0000 7ff13d52bb80 1 -- 172.21.15.16:0/136484371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 msgr2=0x55dbf90d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.110+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff11400bc90 tx=0x7ff11400b470 comp rx=0 tx=0).stop 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.110+0000 7ff13d52bb80 1 -- 172.21.15.16:0/136484371 shutdown_connections 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.110+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d15c0 0x55dbf90d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.111+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.111+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/136484371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90d1080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.111+0000 7ff13d52bb80 1 -- 172.21.15.16:0/136484371 >> 172.21.15.16:0/136484371 conn(0x55dbf8fa9fc0 msgr2=0x55dbf90c38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.111+0000 7ff13d52bb80 1 -- 172.21.15.16:0/136484371 shutdown_connections 2024-05-27T07:02:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.111+0000 7ff13d52bb80 1 -- 172.21.15.16:0/136484371 wait complete. 2024-05-27T07:02:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.112+0000 7ff13d52bb80 1 -- start start 2024-05-27T07:02:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.112+0000 7ff13d52bb80 1 Processor -- start 2024-05-27T07:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8eecfa0 con 0x55dbf90d74e0 2024-05-27T07:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8f87760 con 0x55dbf90d15c0 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dbf90ddfc0 con 0x55dbf8f49e40 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff13d52bb80 10 client.0 _ll_register_callbacks cb 0x55dbf8f90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff12ca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41836/0 (socket says 172.21.15.16:41836) 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff12ca00700 1 -- 172.21.15.16:0/4208928823 learned_addr learned my addr 172.21.15.16:0/4208928823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff127400700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.113+0000 7ff127e00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.114+0000 7ff12ca00700 1 -- 172.21.15.16:0/4208928823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 msgr2=0x55dbf90dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.114+0000 7ff12ca00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.114+0000 7ff12ca00700 1 -- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 msgr2=0x55dbf90db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.114+0000 7ff12ca00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.114+0000 7ff12ca00700 1 -- 172.21.15.16:0/4208928823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf8f368d0 con 0x55dbf90d74e0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.115+0000 7ff12ca00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff1200df3a0 tx=0x7ff1200b2990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da2229cc896217f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.115+0000 7ff112000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.115+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200a9850 con 0x55dbf90d74e0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.115+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff1200c43c0 con 0x55dbf90d74e0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.115+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1140044c0 con 0x55dbf90d74e0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1200c9050 con 0x55dbf90d74e0 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 10 client.15627 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dbf8fb7060 con 0x55dbf90d74e0 2024-05-27T07:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1140044c0 con 0x55dbf90d74e0 2024-05-27T07:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff1200a56b0 con 0x55dbf90d74e0 2024-05-27T07:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 20 client.15627 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf90eae80 0x55dbf90ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 4 client.15627 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.116+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf8ff8d90 con 0x55dbf90eae80 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.117+0000 7ff127e00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf90eae80 0x55dbf90ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.118+0000 7ff127e00700 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf90eae80 0x55dbf90ed2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45c4975dc47644e server_cookie=d4c2401ee6f9d0b0 in_seq=0 out_seq=0 2024-05-27T07:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.118+0000 7ff112000700 10 client.15627 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff112000700 1 -- 172.21.15.16:0/4208928823 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dbf8ff8d90 con 0x55dbf90eae80 2024-05-27T07:02:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff112000700 10 client.15627 handle_command_reply: tid=0 2024-05-27T07:02:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 client.15627 shutdown 2024-05-27T07:02:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 msgr2=0x55dbf90db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff1200df3a0 tx=0x7ff1200b2990 comp rx=0 tx=0).stop 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 shutdown_connections 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf90eae80 0x55dbf90ed2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d74e0 0x55dbf90db770 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/4208928823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 >> 172.21.15.16:0/4208928823 conn(0x55dbf8fa9fc0 msgr2=0x55dbf90c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.121+0000 7ff112000700 0 client.15627 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.122+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 shutdown_connections 2024-05-27T07:02:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.122+0000 7ff13d52bb80 1 -- 172.21.15.16:0/4208928823 wait complete. 2024-05-27T07:02:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.122+0000 7ff13d52bb80 20 client.15627 trim_cache size 0 max 16384 2024-05-27T07:02:21.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.153+0000 7ff13d52bb80 1 Processor -- start 2024-05-27T07:02:21.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.153+0000 7ff13d52bb80 1 -- start start 2024-05-27T07:02:21.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d74e0 0x55dbf90d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff10800ee00 con 0x55dbf8f49e40 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff108010940 con 0x55dbf90d15c0 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff10800f4d0 con 0x55dbf90d74e0 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.154+0000 7ff127e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90dd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41842/0 (socket says 172.21.15.16:41842) 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127e00700 1 -- 172.21.15.16:0/1360910546 learned_addr learned my addr 172.21.15.16:0/1360910546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff12ca00700 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d74e0 0x55dbf90d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 -- 172.21.15.16:0/1360910546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d74e0 msgr2=0x55dbf90d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d74e0 0x55dbf90d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 -- 172.21.15.16:0/1360910546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 msgr2=0x55dbf90dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90dd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.155+0000 7ff127400700 1 -- 172.21.15.16:0/1360910546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf90dbae0 con 0x55dbf90d15c0 2024-05-27T07:02:21.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.156+0000 7ff127400700 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90d6560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff11400d010 tx=0x7ff11400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ee2ea602b9cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.157+0000 7ff11cc00700 1 -- 172.21.15.16:0/1360910546 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400acb0 con 0x55dbf90d15c0 2024-05-27T07:02:21.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.157+0000 7ff11cc00700 1 -- 172.21.15.16:0/1360910546 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff11400ae50 con 0x55dbf90d15c0 2024-05-27T07:02:21.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.157+0000 7ff11cc00700 1 -- 172.21.15.16:0/1360910546 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114004cf0 con 0x55dbf90d15c0 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.158+0000 7ff13d52bb80 1 -- 172.21.15.16:0/1360910546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 msgr2=0x55dbf90d6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.158+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90d6560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff11400d010 tx=0x7ff11400b870 comp rx=0 tx=0).stop 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 -- 172.21.15.16:0/1360910546 shutdown_connections 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf90d74e0 0x55dbf90d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf8f49e40 0x55dbf90dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/1360910546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d15c0 0x55dbf90d6560 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 -- 172.21.15.16:0/1360910546 >> 172.21.15.16:0/1360910546 conn(0x55dbf8fa9fc0 msgr2=0x55dbf90d5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 -- 172.21.15.16:0/1360910546 shutdown_connections 2024-05-27T07:02:21.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.159+0000 7ff13d52bb80 1 -- 172.21.15.16:0/1360910546 wait complete. 2024-05-27T07:02:21.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.160+0000 7ff13d52bb80 1 -- start start 2024-05-27T07:02:21.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.160+0000 7ff13d52bb80 1 Processor -- start 2024-05-27T07:02:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 0x55dbf90e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff108010270 con 0x55dbf90d15c0 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff10800fba0 con 0x55dbf90d74e0 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff10800cda0 con 0x55dbf8f49e40 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff127e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff13d52bb80 10 client.0 _ll_register_callbacks cb 0x55dbf8f90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.161+0000 7ff127e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33404/0 (socket says 172.21.15.16:33404) 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127e00700 1 -- 172.21.15.16:0/595475839 learned_addr learned my addr 172.21.15.16:0/595475839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127400700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 0x55dbf90e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff12ca00700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127e00700 1 -- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 msgr2=0x55dbf90f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127e00700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127e00700 1 -- 172.21.15.16:0/595475839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 msgr2=0x55dbf90e3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.162+0000 7ff127e00700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 0x55dbf90e3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.163+0000 7ff127e00700 1 -- 172.21.15.16:0/595475839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf90f2650 con 0x55dbf8f49e40 2024-05-27T07:02:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.163+0000 7ff127400700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 0x55dbf90e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:02:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.163+0000 7ff127e00700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff118019000 tx=0x7ff118009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af74c13d76ca448 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.163+0000 7ff112000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:02:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118005040 con 0x55dbf8f49e40 2024-05-27T07:02:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11400ae50 con 0x55dbf8f49e40 2024-05-27T07:02:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff1180051e0 con 0x55dbf8f49e40 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1180053e0 con 0x55dbf8f49e40 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff13d52bb80 10 client.25339 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.164+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dbf9155390 con 0x55dbf8f49e40 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff11400ae50 con 0x55dbf8f49e40 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff118004690 con 0x55dbf8f49e40 2024-05-27T07:02:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 10 client.25339 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 10 client.25339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 20 client.25339 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf9155a50 0x55dbf9157e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 4 client.25339 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.165+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbf8ff2570 con 0x55dbf9155a50 2024-05-27T07:02:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.166+0000 7ff127400700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf9155a50 0x55dbf9157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.167+0000 7ff127400700 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf9155a50 0x55dbf9157e70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c84530054de7b9 server_cookie=163f8ba4f2d4252b in_seq=0 out_seq=0 2024-05-27T07:02:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.167+0000 7ff112000700 10 client.25339 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.167+0000 7ff112000700 1 -- 172.21.15.16:0/595475839 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbf8ff2570 con 0x55dbf9155a50 2024-05-27T07:02:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff112000700 10 client.25339 handle_command_reply: tid=0 2024-05-27T07:02:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 client.25339 shutdown 2024-05-27T07:02:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 msgr2=0x55dbf90e34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff118019000 tx=0x7ff118009310 comp rx=0 tx=0).stop 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 shutdown_connections 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55dbf9155a50 0x55dbf9157e70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbf90d74e0 0x55dbf90f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dbf8f49e40 0x55dbf90e34e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 --2- 172.21.15.16:0/595475839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf90d15c0 0x55dbf90e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 >> 172.21.15.16:0/595475839 conn(0x55dbf8fa9fc0 msgr2=0x55dbf90c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff112000700 0 client.25339 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.168+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 shutdown_connections 2024-05-27T07:02:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.169+0000 7ff13d52bb80 1 -- 172.21.15.16:0/595475839 wait complete. 2024-05-27T07:02:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.169+0000 7ff13d52bb80 20 client.25339 trim_cache size 0 max 16384 2024-05-27T07:02:21.162 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 msgr2=0x7ff10c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 0x7ff10c0b51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff11801a040 tx=0x7ff118009920 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 msgr2=0x7ff128198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff120007ee0 tx=0x7ff12000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 shutdown_connections 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff10c0b2ca0 0x7ff10c0b51b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff128108730 0x7ff128198910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1281034b0 0x7ff12819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 --2- 172.21.15.16:0/2228678771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff128107de0 0x7ff12819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 >> 172.21.15.16:0/2228678771 conn(0x7ff1280fedb0 msgr2=0x7ff128109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.170+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 shutdown_connections 2024-05-27T07:02:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:21.171+0000 7ff12ea00700 1 -- 172.21.15.16:0/2228678771 wait complete. 2024-05-27T07:02:21.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:21 smithi164 ceph-mon[35327]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-27T07:02:21.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:21 smithi016 ceph-mon[31415]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-27T07:02:21.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:21 smithi069 ceph-mon[35276]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-27T07:02:23.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:02:23.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:02:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:23.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:23 smithi016 ceph-mon[31415]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:23.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:23 smithi164 ceph-mon[35327]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:23 smithi069 ceph-mon[35276]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:25.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:25 smithi164 ceph-mon[35327]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:25.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:25 smithi016 ceph-mon[31415]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:25.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:25 smithi069 ceph-mon[35276]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.178+0000 7f0033a00700 1 -- 172.21.15.16:0/247670560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c109860 msgr2=0x7f002c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.178+0000 7f0033a00700 1 --2- 172.21.15.16:0/247670560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c109860 0x7f002c1111c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f002400b1d0 tx=0x7f002400b4e0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 -- 172.21.15.16:0/247670560 shutdown_connections 2024-05-27T07:02:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 --2- 172.21.15.16:0/247670560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c109860 0x7f002c1111c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 --2- 172.21.15.16:0/247670560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 0x7f002c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 --2- 172.21.15.16:0/247670560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c108f10 0x7f002c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 -- 172.21.15.16:0/247670560 >> 172.21.15.16:0/247670560 conn(0x7f002c100030 msgr2=0x7f002c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 -- 172.21.15.16:0/247670560 shutdown_connections 2024-05-27T07:02:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.179+0000 7f0033a00700 1 -- 172.21.15.16:0/247670560 wait complete. 2024-05-27T07:02:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.180+0000 7f0033a00700 1 Processor -- start 2024-05-27T07:02:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 -- start start 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 0x7f002c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c109860 0x7f002c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f002c112210 con 0x7f002c108f10 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f002c112090 con 0x7f002c109860 2024-05-27T07:02:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.181+0000 7f0033a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f002c112390 con 0x7f002c1045e0 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.182+0000 7f002be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.182+0000 7f002be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51830/0 (socket says 172.21.15.16:51830) 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.182+0000 7f002be00700 1 -- 172.21.15.16:0/1141213588 learned_addr learned my addr 172.21.15.16:0/1141213588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.182+0000 7f0031a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c109860 0x7f002c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.182+0000 7f0031000700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 0x7f002c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 -- 172.21.15.16:0/1141213588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 msgr2=0x7f002c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 0x7f002c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 -- 172.21.15.16:0/1141213588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c109860 msgr2=0x7f002c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c109860 0x7f002c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 -- 172.21.15.16:0/1141213588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c199000 con 0x7f002c108f10 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.183+0000 7f002be00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f001800d980 tx=0x7f001800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.184+0000 7f0029600700 1 -- 172.21.15.16:0/1141213588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018017070 con 0x7f002c108f10 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.184+0000 7f0029600700 1 -- 172.21.15.16:0/1141213588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0018009dc0 con 0x7f002c108f10 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.184+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002c1992f0 con 0x7f002c108f10 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.184+0000 7f0029600700 1 -- 172.21.15.16:0/1141213588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001800f3c0 con 0x7f002c108f10 2024-05-27T07:02:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.184+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002c1a55c0 con 0x7f002c108f10 2024-05-27T07:02:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.190+0000 7f0029600700 1 -- 172.21.15.16:0/1141213588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 149936+0+0 (secure 0 0 0) 0x7f0018009930 con 0x7f002c108f10 2024-05-27T07:02:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.190+0000 7f0042568b80 1 Processor -- start 2024-05-27T07:02:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.190+0000 7f0042568b80 1 -- start start 2024-05-27T07:02:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe1b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfd49ec0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfd39a60 con 0x55c4cfe21db0 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0042568b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfd3daa0 con 0x55c4cfe1be30 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0031a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0031a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51844/0 (socket says 172.21.15.16:51844) 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.191+0000 7f0031a00700 1 -- 172.21.15.16:0/538864081 learned_addr learned my addr 172.21.15.16:0/538864081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f002be00700 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 -- 172.21.15.16:0/538864081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 msgr2=0x55c4cfe1c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe1c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 -- 172.21.15.16:0/538864081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 msgr2=0x55c4cfe1b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe1b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 -- 172.21.15.16:0/538864081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cfd024a0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0029600700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 0x7f00140b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0031a00700 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0024009cd0 tx=0x7f002400b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57616cd69cdc99bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.192+0000 7f0029600700 1 -- 172.21.15.16:0/1141213588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f00180e2690 con 0x7f002c108f10 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.193+0000 7f001e000700 1 -- 172.21.15.16:0/538864081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024027070 con 0x55c4cfc9cec0 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.193+0000 7f001e000700 1 -- 172.21.15.16:0/538864081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f002400bdc0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.193+0000 7f001e000700 1 -- 172.21.15.16:0/538864081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003e10 con 0x55c4cfc9cec0 2024-05-27T07:02:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0031000700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 0x7f00140b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 -- 172.21.15.16:0/538864081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 msgr2=0x55c4cfe21870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0024009cd0 tx=0x7f002400b140 comp rx=0 tx=0).stop 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 -- 172.21.15.16:0/538864081 shutdown_connections 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe1c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe21870 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 --2- 172.21.15.16:0/538864081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe1b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.194+0000 7f0042568b80 1 -- 172.21.15.16:0/538864081 >> 172.21.15.16:0/538864081 conn(0x55c4cfcfd040 msgr2=0x55c4cfe16950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.195+0000 7f0042568b80 1 -- 172.21.15.16:0/538864081 shutdown_connections 2024-05-27T07:02:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.195+0000 7f0031000700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 0x7f00140b5130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0020005950 tx=0x7f00200058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.195+0000 7f0042568b80 1 -- 172.21.15.16:0/538864081 wait complete. 2024-05-27T07:02:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.196+0000 7f0042568b80 1 -- start start 2024-05-27T07:02:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.196+0000 7f0042568b80 1 Processor -- start 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfc40020 con 0x55c4cfc9cec0 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfcda7e0 con 0x55c4cfe1be30 2024-05-27T07:02:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4cfe28820 con 0x55c4cfe21db0 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0042568b80 10 client.0 _ll_register_callbacks cb 0x55c4cfce3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0031000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0031000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51860/0 (socket says 172.21.15.16:51860) 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f0031000700 1 -- 172.21.15.16:0/2216471214 learned_addr learned my addr 172.21.15.16:0/2216471214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.197+0000 7f002be00700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031a00700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 -- 172.21.15.16:0/2216471214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 msgr2=0x55c4cfe26010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe26010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 -- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 msgr2=0x55c4cfe25ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe25ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 -- 172.21.15.16:0/2216471214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cfc89950 con 0x55c4cfc9cec0 2024-05-27T07:02:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0031000700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0020005e10 tx=0x7f0020004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811deb532213cefa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.198+0000 7f0012a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020022070 con 0x55c4cfc9cec0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f002001e650 con 0x55c4cfc9cec0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f002400bdc0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f002002c050 con 0x55c4cfc9cec0 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0042568b80 10 client.15645 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.199+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4cfd0a0e0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.200+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f002400bdc0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.200+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f00200277b0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.200+0000 7f0042568b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.200+0000 7f0042568b80 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.200+0000 7f0042568b80 20 client.15645 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.201+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfe357e0 0x55c4cfe37cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.201+0000 7f0042568b80 4 client.15645 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.201+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4cfd4be10 con 0x55c4cfe357e0 2024-05-27T07:02:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.201+0000 7f002be00700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfe357e0 0x55c4cfe37cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.202+0000 7f002be00700 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfe357e0 0x55c4cfe37cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b000867f2542e4 server_cookie=1b44a5b482a3a14c in_seq=0 out_seq=0 2024-05-27T07:02:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.202+0000 7f0012a00700 10 client.15645 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.204+0000 7f0012a00700 1 -- 172.21.15.16:0/2216471214 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c4cfd4be10 con 0x55c4cfe357e0 2024-05-27T07:02:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.204+0000 7f0012a00700 10 client.15645 handle_command_reply: tid=0 2024-05-27T07:02:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.204+0000 7f0042568b80 1 client.15645 shutdown 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.204+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 msgr2=0x55c4cfe25590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.204+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0020005e10 tx=0x7f0020004750 comp rx=0 tx=0).stop 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 shutdown_connections 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfe357e0 0x55c4cfe37cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe26010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe25590 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 --2- 172.21.15.16:0/2216471214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 >> 172.21.15.16:0/2216471214 conn(0x55c4cfcfd040 msgr2=0x55c4cfe16600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0012a00700 0 client.15645 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 shutdown_connections 2024-05-27T07:02:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 1 -- 172.21.15.16:0/2216471214 wait complete. 2024-05-27T07:02:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.205+0000 7f0042568b80 20 client.15645 trim_cache size 0 max 16384 2024-05-27T07:02:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 Processor -- start 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 -- start start 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe1b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe25bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1ae240 con 0x55c4cfc9cec0 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1afd80 con 0x55c4cfe1be30 2024-05-27T07:02:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.230+0000 7f0042568b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1ae910 con 0x55c4cfe21db0 2024-05-27T07:02:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.231+0000 7f002be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.231+0000 7f002be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41248/0 (socket says 172.21.15.16:41248) 2024-05-27T07:02:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.231+0000 7f002be00700 1 -- 172.21.15.16:0/599740213 learned_addr learned my addr 172.21.15.16:0/599740213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.231+0000 7f0031a00700 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe25bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.231+0000 7f0031000700 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe1b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 -- 172.21.15.16:0/599740213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 msgr2=0x55c4cfe25bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe25bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 -- 172.21.15.16:0/599740213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 msgr2=0x55c4cfe1b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe1b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 -- 172.21.15.16:0/599740213 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cfe33200 con 0x55c4cfe1be30 2024-05-27T07:02:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.232+0000 7f002be00700 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00180a6610 tx=0x7f0018005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2943b3cd92ccf89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.233+0000 7f001d600700 1 -- 172.21.15.16:0/599740213 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018020040 con 0x55c4cfe1be30 2024-05-27T07:02:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.233+0000 7f001d600700 1 -- 172.21.15.16:0/599740213 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00180a8eb0 con 0x55c4cfe1be30 2024-05-27T07:02:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.233+0000 7f001d600700 1 -- 172.21.15.16:0/599740213 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018010680 con 0x55c4cfe1be30 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 -- 172.21.15.16:0/599740213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 msgr2=0x55c4cfe31880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00180a6610 tx=0x7f0018005c40 comp rx=0 tx=0).stop 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 -- 172.21.15.16:0/599740213 shutdown_connections 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe21db0 0x55c4cfe25bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe1b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 --2- 172.21.15.16:0/599740213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe1be30 0x55c4cfe31880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 -- 172.21.15.16:0/599740213 >> 172.21.15.16:0/599740213 conn(0x55c4cfcfd040 msgr2=0x55c4cfe22ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 -- 172.21.15.16:0/599740213 shutdown_connections 2024-05-27T07:02:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.234+0000 7f0042568b80 1 -- 172.21.15.16:0/599740213 wait complete. 2024-05-27T07:02:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.235+0000 7f0042568b80 1 -- start start 2024-05-27T07:02:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.235+0000 7f0042568b80 1 Processor -- start 2024-05-27T07:02:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe2aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe2af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1af6b0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1aefe0 con 0x55c4cfe21db0 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1ac1e0 con 0x55c4cfe1be30 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0042568b80 10 client.0 _ll_register_callbacks cb 0x55c4cfce3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0031000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0031000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51878/0 (socket says 172.21.15.16:51878) 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.236+0000 7f0031000700 1 -- 172.21.15.16:0/2360420075 learned_addr learned my addr 172.21.15.16:0/2360420075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f002be00700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe2aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 -- 172.21.15.16:0/2360420075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 msgr2=0x55c4cfe2aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe2aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 -- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 msgr2=0x55c4cfe2af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe2af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 -- 172.21.15.16:0/2360420075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cfe2b4c0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.237+0000 7f0031000700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0020027d60 tx=0x7f0020011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19ff19db0e33a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0012a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020022070 con 0x55c4cfc9cec0 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f0020032960 con 0x55c4cfc9cec0 2024-05-27T07:02:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00180a8eb0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0020032b60 con 0x55c4cfc9cec0 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0042568b80 10 client.15651 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.238+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4cfea9830 con 0x55c4cfc9cec0 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00180a8eb0 con 0x55c4cfc9cec0 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f002001e450 con 0x55c4cfc9cec0 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 20 client.15651 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfea9ef0 0x55c4cfeac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 4 client.15651 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:02:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.239+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4cfd455f0 con 0x55c4cfea9ef0 2024-05-27T07:02:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.240+0000 7f002be00700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfea9ef0 0x55c4cfeac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.241+0000 7f002be00700 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfea9ef0 0x55c4cfeac310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed8e14185f96891 server_cookie=6583afa93af5f239 in_seq=0 out_seq=0 2024-05-27T07:02:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.241+0000 7f0012a00700 10 client.15651 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.243+0000 7f0012a00700 1 -- 172.21.15.16:0/2360420075 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4cfd455f0 con 0x55c4cfea9ef0 2024-05-27T07:02:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.243+0000 7f0012a00700 10 client.15651 handle_command_reply: tid=0 2024-05-27T07:02:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.243+0000 7f0042568b80 1 client.15651 shutdown 2024-05-27T07:02:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 msgr2=0x55c4cfe2de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0020027d60 tx=0x7f0020011ab0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 shutdown_connections 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55c4cfea9ef0 0x55c4cfeac310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4cfe21db0 0x55c4cfe2af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4cfc9cec0 0x55c4cfe2de70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 --2- 172.21.15.16:0/2360420075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cfe1be30 0x55c4cfe2aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0012a00700 0 client.15651 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.244+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 >> 172.21.15.16:0/2360420075 conn(0x55c4cfcfd040 msgr2=0x55c4cfe22ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.245+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 shutdown_connections 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.245+0000 7f0042568b80 1 -- 172.21.15.16:0/2360420075 wait complete. 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.245+0000 7f0042568b80 20 client.15651 trim_cache size 0 max 16384 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:02:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c46481aa-a908-433c-b806-8f87ec499016", 2024-05-27T07:02:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:02:26.239 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:02:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 msgr2=0x7f00140b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 0x7f00140b5130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0020005950 tx=0x7f00200058e0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 msgr2=0x7f002c19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f001800d980 tx=0x7f001800dc90 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 shutdown_connections 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00140b2c20 0x7f00140b5130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f002c109860 0x7f002c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f002c1045e0 0x7f002c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 --2- 172.21.15.16:0/1141213588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c108f10 0x7f002c19ec70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 >> 172.21.15.16:0/1141213588 conn(0x7f002c100030 msgr2=0x7f002c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.246+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 shutdown_connections 2024-05-27T07:02:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:26.247+0000 7f0033a00700 1 -- 172.21.15.16:0/1141213588 wait complete. 2024-05-27T07:02:27.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:27 smithi016 ceph-mon[31415]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:27 smithi164 ceph-mon[35327]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-27T07:02:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:27.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:27.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:02:27.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:02:27.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:02:27.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:27 smithi069 ceph-mon[35276]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-27T07:02:29.781 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:29 smithi016 ceph-mon[31415]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:29.781 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:29 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:02:29.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:29 smithi164 ceph-mon[35327]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:29.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:29 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:02:29.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:29 smithi069 ceph-mon[35276]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:29.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:29 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2898840327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 msgr2=0x7fb9b8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2898840327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b8104800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80099c0 tx=0x7fb9a8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2898840327 shutdown_connections 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2898840327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 0x7fb9b8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2898840327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b8104800 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2898840327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.514+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2898840327 >> 172.21.15.16:0/2898840327 conn(0x7fb9b8079d00 msgr2=0x7fb9b807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.515+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2898840327 shutdown_connections 2024-05-27T07:02:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.515+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2898840327 wait complete. 2024-05-27T07:02:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.517+0000 7fb9b7e00700 1 Processor -- start 2024-05-27T07:02:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.517+0000 7fb9b7e00700 1 -- start start 2024-05-27T07:02:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.517+0000 7fb9b7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 0x7fb9b819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81141c0 con 0x7fb9b81043f0 2024-05-27T07:02:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8114040 con 0x7fb9b8108d20 2024-05-27T07:02:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8114340 con 0x7fb9b8109670 2024-05-27T07:02:30.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51896/0 (socket says 172.21.15.16:51896) 2024-05-27T07:02:30.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 0x7fb9b819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b6a00700 1 -- 172.21.15.16:0/2020160738 learned_addr learned my addr 172.21.15.16:0/2020160738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:30.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.518+0000 7fb9b6000700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 -- 172.21.15.16:0/2020160738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 msgr2=0x7fb9b819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 0x7fb9b819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 -- 172.21.15.16:0/2020160738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 msgr2=0x7fb9b81a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b81a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 -- 172.21.15.16:0/2020160738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b819b0d0 con 0x7fb9b81043f0 2024-05-27T07:02:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6000700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b81a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.520+0000 7fb9b6a00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b580 tx=0x7fb9a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.521+0000 7fb9bca00700 1 -- 172.21.15.16:0/2020160738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8027070 con 0x7fb9b81043f0 2024-05-27T07:02:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.521+0000 7fb9bca00700 1 -- 172.21.15.16:0/2020160738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9a800fd30 con 0x7fb9b81043f0 2024-05-27T07:02:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.521+0000 7fb9bca00700 1 -- 172.21.15.16:0/2020160738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8022e50 con 0x7fb9b81043f0 2024-05-27T07:02:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.521+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b819b360 con 0x7fb9b81043f0 2024-05-27T07:02:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.521+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b819b7f0 con 0x7fb9b81043f0 2024-05-27T07:02:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.528+0000 7fb9cc89ab80 1 Processor -- start 2024-05-27T07:02:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 -- start start 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9bca00700 1 -- 172.21.15.16:0/2020160738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb9a802a070 con 0x7fb9b81043f0 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd65104e40 0x55cd65284c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd652906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd651b1e40 con 0x55cd651f09f0 2024-05-27T07:02:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd651a19e0 con 0x55cd65104e40 2024-05-27T07:02:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.529+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd651a5a20 con 0x55cd6528e2c0 2024-05-27T07:02:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.530+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.530+0000 7fb9b6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd65104e40 0x55cd65284c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51908/0 (socket says 172.21.15.16:51908) 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b7400700 1 -- 172.21.15.16:0/3131705841 learned_addr learned my addr 172.21.15.16:0/3131705841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b6000700 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd652906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b7400700 1 -- 172.21.15.16:0/3131705841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 msgr2=0x55cd652906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd652906e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.531+0000 7fb9b7400700 1 -- 172.21.15.16:0/3131705841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd65104e40 msgr2=0x55cd65284c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.532+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd65104e40 0x55cd65284c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.532+0000 7fb9b7400700 1 -- 172.21.15.16:0/3131705841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6516a420 con 0x55cd651f09f0 2024-05-27T07:02:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.532+0000 7fb9bca00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 0x7fb99c0b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.532+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0003fa0 tx=0x7fb9b0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb0d80713adbc8f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.532+0000 7fb9bca00700 1 -- 172.21.15.16:0/2020160738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb9a80e3570 con 0x7fb9b81043f0 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.533+0000 7fb9b6a00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 0x7fb99c0b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.533+0000 7fb9acc00700 1 -- 172.21.15.16:0/3131705841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0004bb0 con 0x55cd651f09f0 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.533+0000 7fb9acc00700 1 -- 172.21.15.16:0/3131705841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9b00048e0 con 0x55cd651f09f0 2024-05-27T07:02:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.533+0000 7fb9acc00700 1 -- 172.21.15.16:0/3131705841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0007d20 con 0x55cd651f09f0 2024-05-27T07:02:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3131705841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 msgr2=0x55cd65284750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0003fa0 tx=0x7fb9b0009ba0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9b6a00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 0x7fb99c0b51f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80e09e0 tx=0x7fb9a80e6f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3131705841 shutdown_connections 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd652906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd65284750 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3131705841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd65104e40 0x55cd65284c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3131705841 >> 172.21.15.16:0/3131705841 conn(0x55cd6519e390 msgr2=0x55cd6527f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3131705841 shutdown_connections 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.534+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3131705841 wait complete. 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.535+0000 7fb9cc89ab80 1 -- start start 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.535+0000 7fb9cc89ab80 1 Processor -- start 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd6528b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd6528bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd650a7fa0 con 0x55cd651f09f0 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd65142760 con 0x55cd6528e2c0 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd65299610 con 0x55cd65104e40 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9cc89ab80 10 client.0 _ll_register_callbacks cb 0x55cd6514bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44464/0 (socket says 172.21.15.16:44464) 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.536+0000 7fb9b7400700 1 -- 172.21.15.16:0/3644739228 learned_addr learned my addr 172.21.15.16:0/3644739228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd6528bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b6000700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd6528b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 -- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 msgr2=0x55cd6528bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd6528bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 -- 172.21.15.16:0/3644739228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 msgr2=0x55cd6528b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd6528b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.537+0000 7fb9b7400700 1 -- 172.21.15.16:0/3644739228 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd650f18d0 con 0x55cd65104e40 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.538+0000 7fb9b7400700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0004720 tx=0x7fb9b000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da92d59851e4539 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.538+0000 7fb9a1600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.538+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00079a0 con 0x55cd65104e40 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.538+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b00048e0 con 0x55cd65104e40 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.538+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb9b0007b40 con 0x55cd65104e40 2024-05-27T07:02:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.539+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb9b0007d60 con 0x55cd65104e40 2024-05-27T07:02:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.539+0000 7fb9cc89ab80 10 client.25369 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.539+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd65172060 con 0x55cd65104e40 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.539+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9b00048e0 con 0x55cd65104e40 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb9b0005000 con 0x55cd65104e40 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 10 client.25369 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 10 client.25369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 20 client.25369 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd652a6470 0x55cd652a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 4 client.25369 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.540+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd651b3d90 con 0x55cd652a6470 2024-05-27T07:02:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.541+0000 7fb9b6000700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd652a6470 0x55cd652a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.542+0000 7fb9b6000700 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd652a6470 0x55cd652a8890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260f583dda97a49d server_cookie=36f41179477ef54a in_seq=0 out_seq=0 2024-05-27T07:02:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.542+0000 7fb9a1600700 10 client.25369 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.544+0000 7fb9a1600700 1 -- 172.21.15.16:0/3644739228 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cd651b3d90 con 0x55cd652a6470 2024-05-27T07:02:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.544+0000 7fb9a1600700 10 client.25369 handle_command_reply: tid=0 2024-05-27T07:02:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.544+0000 7fb9cc89ab80 1 client.25369 shutdown 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.544+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 msgr2=0x55cd6528b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.544+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0004720 tx=0x7fb9b000b040 comp rx=0 tx=0).stop 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 shutdown_connections 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd652a6470 0x55cd652a8890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd6528bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd65104e40 0x55cd6528b2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/3644739228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd651f09f0 0x55cd6528b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 >> 172.21.15.16:0/3644739228 conn(0x55cd6519e390 msgr2=0x55cd652847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9a1600700 0 client.25369 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 shutdown_connections 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/3644739228 wait complete. 2024-05-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.545+0000 7fb9cc89ab80 20 client.25369 trim_cache size 0 max 16384 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 Processor -- start 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 -- start start 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd651f09f0 0x55cd652a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd6527f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81ae040 con 0x55cd65104e40 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81afb80 con 0x55cd651f09f0 2024-05-27T07:02:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.570+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81ae710 con 0x55cd6528e2c0 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51930/0 (socket says 172.21.15.16:51930) 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 -- 172.21.15.16:0/763939054 learned_addr learned my addr 172.21.15.16:0/763939054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 -- 172.21.15.16:0/763939054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 msgr2=0x55cd6527f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd6527f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 -- 172.21.15.16:0/763939054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd651f09f0 msgr2=0x55cd652a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd651f09f0 0x55cd652a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.571+0000 7fb9b7400700 1 -- 172.21.15.16:0/763939054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd652994c0 con 0x55cd65104e40 2024-05-27T07:02:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.572+0000 7fb9b7400700 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009bc0 tx=0x7fb9b0004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f94e6a8b3143e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.573+0000 7fb9a3e00700 1 -- 172.21.15.16:0/763939054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0031040 con 0x55cd65104e40 2024-05-27T07:02:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.573+0000 7fb9a3e00700 1 -- 172.21.15.16:0/763939054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9b0009100 con 0x55cd65104e40 2024-05-27T07:02:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.573+0000 7fb9a3e00700 1 -- 172.21.15.16:0/763939054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0005980 con 0x55cd65104e40 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.574+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/763939054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 msgr2=0x55cd65284e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.574+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009bc0 tx=0x7fb9b0004a60 comp rx=0 tx=0).stop 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/763939054 shutdown_connections 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd6528e2c0 0x55cd6527f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd65284e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/763939054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd651f09f0 0x55cd652a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/763939054 >> 172.21.15.16:0/763939054 conn(0x55cd6519e390 msgr2=0x55cd65283790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/763939054 shutdown_connections 2024-05-27T07:02:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.575+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/763939054 wait complete. 2024-05-27T07:02:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.576+0000 7fb9cc89ab80 1 -- start start 2024-05-27T07:02:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.576+0000 7fb9cc89ab80 1 Processor -- start 2024-05-27T07:02:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 0x55cd652a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd652a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81af4b0 con 0x55cd65104e40 2024-05-27T07:02:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81aede0 con 0x55cd6528e2c0 2024-05-27T07:02:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81abfe0 con 0x55cd651f09f0 2024-05-27T07:02:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.577+0000 7fb9cc89ab80 10 client.0 _ll_register_callbacks cb 0x55cd6514bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51940/0 (socket says 172.21.15.16:51940) 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 -- 172.21.15.16:0/980464542 learned_addr learned my addr 172.21.15.16:0/980464542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b6000700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 0x55cd652a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 -- 172.21.15.16:0/980464542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 msgr2=0x55cd652a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 0x55cd652a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 -- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 msgr2=0x55cd652a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd652a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.578+0000 7fb9b7400700 1 -- 172.21.15.16:0/980464542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd652a9b40 con 0x55cd65104e40 2024-05-27T07:02:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.579+0000 7fb9b6000700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 0x55cd652a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.579+0000 7fb9b7400700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00045b0 tx=0x7fb9b0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563cd40a0e19d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.579+0000 7fb9a1600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.579+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0031070 con 0x55cd65104e40 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.580+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb9b0007b30 con 0x55cd65104e40 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.580+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b0009100 con 0x55cd65104e40 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.580+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb9b0048050 con 0x55cd65104e40 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.580+0000 7fb9cc89ab80 10 client.15666 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.580+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd65310970 con 0x55cd65104e40 2024-05-27T07:02:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9b0009100 con 0x55cd65104e40 2024-05-27T07:02:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb9b003d020 con 0x55cd65104e40 2024-05-27T07:02:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 10 client.15666 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 20 client.15666 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd65310f00 0x55cd65313320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 4 client.15666 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:02:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.581+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd651ad570 con 0x55cd65310f00 2024-05-27T07:02:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.582+0000 7fb9b6000700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd65310f00 0x55cd65313320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.583+0000 7fb9b6000700 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd65310f00 0x55cd65313320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1cec8c01696b9a server_cookie=68b88715b278c060 in_seq=0 out_seq=0 2024-05-27T07:02:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.583+0000 7fb9a1600700 10 client.15666 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.583+0000 7fb9a1600700 1 -- 172.21.15.16:0/980464542 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd651ad570 con 0x55cd65310f00 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.583+0000 7fb9a1600700 10 client.15666 handle_command_reply: tid=0 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.583+0000 7fb9cc89ab80 1 client.15666 shutdown 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 msgr2=0x55cd652abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00045b0 tx=0x7fb9b0009360 comp rx=0 tx=0).stop 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 shutdown_connections 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55cd65310f00 0x55cd65313320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6528e2c0 0x55cd652a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd65104e40 0x55cd652abd40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 --2- 172.21.15.16:0/980464542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd651f09f0 0x55cd652a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 >> 172.21.15.16:0/980464542 conn(0x55cd6519e390 msgr2=0x55cd6528b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.584+0000 7fb9a1600700 0 client.15666 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.585+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 shutdown_connections 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.585+0000 7fb9cc89ab80 1 -- 172.21.15.16:0/980464542 wait complete. 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.585+0000 7fb9cc89ab80 20 client.15666 trim_cache size 0 max 16384 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:02:30.578 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 msgr2=0x7fb99c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 0x7fb99c0b51f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80e09e0 tx=0x7fb9a80e6f90 comp rx=0 tx=0).stop 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 msgr2=0x7fb9b81a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b580 tx=0x7fb9a80047b0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 shutdown_connections 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb99c0b2ce0 0x7fb99c0b51f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b8109670 0x7fb9b819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b81043f0 0x7fb9b81a0800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 --2- 172.21.15.16:0/2020160738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8108d20 0x7fb9b81a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.586+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 >> 172.21.15.16:0/2020160738 conn(0x7fb9b8079d00 msgr2=0x7fb9b8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.587+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 shutdown_connections 2024-05-27T07:02:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:30.587+0000 7fb9b7e00700 1 -- 172.21.15.16:0/2020160738 wait complete. 2024-05-27T07:02:31.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:02:31.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:02:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:31 smithi164 ceph-mon[35327]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-27T07:02:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:31 smithi016 ceph-mon[31415]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-27T07:02:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:31 smithi069 ceph-mon[35276]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-27T07:02:32.098 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:33.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:33 smithi016 ceph-mon[31415]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-27T07:02:33.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:33 smithi164 ceph-mon[35327]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-27T07:02:33.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:33 smithi069 ceph-mon[35276]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-27T07:02:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 -- 172.21.15.16:0/1674767393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 msgr2=0x7f54a8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 --2- 172.21.15.16:0/1674767393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a8104800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f549c0099c0 tx=0x7f549c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 -- 172.21.15.16:0/1674767393 shutdown_connections 2024-05-27T07:02:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 --2- 172.21.15.16:0/1674767393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8109670 0x7f54a8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 --2- 172.21.15.16:0/1674767393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a8104800 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 --2- 172.21.15.16:0/1674767393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8108d20 0x7f54a8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.753+0000 7f54ae200700 1 -- 172.21.15.16:0/1674767393 >> 172.21.15.16:0/1674767393 conn(0x7f54a8079d00 msgr2=0x7f54a807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.754+0000 7f54ae200700 1 -- 172.21.15.16:0/1674767393 shutdown_connections 2024-05-27T07:02:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.754+0000 7f54ae200700 1 -- 172.21.15.16:0/1674767393 wait complete. 2024-05-27T07:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.755+0000 7f54ae200700 1 Processor -- start 2024-05-27T07:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.756+0000 7f54ae200700 1 -- start start 2024-05-27T07:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.756+0000 7f54ae200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ae200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 0x7f54a819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ae200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8109670 0x7f54a81a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ae200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81141a0 con 0x7f54a81043f0 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ae200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8114020 con 0x7f54a8108d20 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ae200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8114320 con 0x7f54a8109670 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ace00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ace00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52534/0 (socket says 172.21.15.16:52534) 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.757+0000 7f54ace00700 1 -- 172.21.15.16:0/644811142 learned_addr learned my addr 172.21.15.16:0/644811142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ad800700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8109670 0x7f54a81a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ace00700 1 -- 172.21.15.16:0/644811142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8109670 msgr2=0x7f54a81a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ace00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8109670 0x7f54a81a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ace00700 1 -- 172.21.15.16:0/644811142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 msgr2=0x7f54a819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54a7e00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 0x7f54a819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ace00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 0x7f54a819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54ace00700 1 -- 172.21.15.16:0/644811142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81a1d20 con 0x7f54a81043f0 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.758+0000 7f54a7e00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 0x7f54a819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.759+0000 7f54ace00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f50 tx=0x7f549c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.760+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c027070 con 0x7f54a81043f0 2024-05-27T07:02:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.760+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f549c00fdd0 con 0x7f54a81043f0 2024-05-27T07:02:34.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.760+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a81a1fb0 con 0x7f54a81043f0 2024-05-27T07:02:34.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.760+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a9930 con 0x7f54a81043f0 2024-05-27T07:02:34.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.760+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c0173f0 con 0x7f54a81043f0 2024-05-27T07:02:34.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.761+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5488005330 con 0x7f54a81043f0 2024-05-27T07:02:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.766+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f549c00f940 con 0x7f54a81043f0 2024-05-27T07:02:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.768+0000 7f54a5600700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 0x7f54900b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.769+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f549c0e3a40 con 0x7f54a81043f0 2024-05-27T07:02:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.769+0000 7f54a7e00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 0x7f54900b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.771+0000 7f54a7e00700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 0x7f54900b5000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f5494005ee0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:34.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:34.773+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f549c013070 con 0x7f54a81043f0 2024-05-27T07:02:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.289+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5488005620 con 0x7f54a81043f0 2024-05-27T07:02:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.290+0000 7f54a5600700 1 -- 172.21.15.16:0/644811142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f549c0ab0c0 con 0x7f54a81043f0 2024-05-27T07:02:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:02:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.293+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 msgr2=0x7f54900b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.293+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 0x7f54900b5000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f5494005ee0 comp rx=0 tx=0).stop 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.293+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 msgr2=0x7f54a819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.293+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f50 tx=0x7f549c00f510 comp rx=0 tx=0).stop 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 shutdown_connections 2024-05-27T07:02:35.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f54900b2af0 0x7f54900b5000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54a8109670 0x7f54a81a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a81043f0 0x7f54a819ce50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 --2- 172.21.15.16:0/644811142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54a8108d20 0x7f54a819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 >> 172.21.15.16:0/644811142 conn(0x7f54a8079d00 msgr2=0x7f54a8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 shutdown_connections 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:35.294+0000 7f54ae200700 1 -- 172.21.15.16:0/644811142 wait complete. 2024-05-27T07:02:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:02:35.833 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:35 smithi164 ceph-mon[35327]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:35.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:35 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/644811142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:02:35.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:35 smithi016 ceph-mon[31415]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:35.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:35 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/644811142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:02:35.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:35 smithi069 ceph-mon[35276]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T07:02:35.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:35 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/644811142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:02:36.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:02:36.784 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:37.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:37 smithi164 ceph-mon[35327]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:37.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:37 smithi016 ceph-mon[31415]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:37.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:37 smithi069 ceph-mon[35276]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-27T07:02:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 -- 172.21.15.16:0/2706061406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741084c0 msgr2=0x7fd374108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 --2- 172.21.15.16:0/2706061406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741084c0 0x7fd374108990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b0a0 tx=0x7fd37000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 -- 172.21.15.16:0/2706061406 shutdown_connections 2024-05-27T07:02:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 --2- 172.21.15.16:0/2706061406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741084c0 0x7fd374108990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 --2- 172.21.15.16:0/2706061406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 0x7fd374103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 --2- 172.21.15.16:0/2706061406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741039c0 0x7fd374107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 -- 172.21.15.16:0/2706061406 >> 172.21.15.16:0/2706061406 conn(0x7fd374078ab0 msgr2=0x7fd374078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 -- 172.21.15.16:0/2706061406 shutdown_connections 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.395+0000 7fd37b200700 1 -- 172.21.15.16:0/2706061406 wait complete. 2024-05-27T07:02:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.397+0000 7fd37b200700 1 Processor -- start 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.397+0000 7fd37b200700 1 -- start start 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.397+0000 7fd37b200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 0x7fd3741a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.397+0000 7fd37b200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.397+0000 7fd37b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 0x7fd3741a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd37b200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd37411a750 con 0x7fd3741039c0 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd37b200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd37411a5d0 con 0x7fd374103070 2024-05-27T07:02:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd37b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd37411a8d0 con 0x7fd3741084c0 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52548/0 (socket says 172.21.15.16:52548) 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd379400700 1 -- 172.21.15.16:0/2729317224 learned_addr learned my addr 172.21.15.16:0/2729317224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd379e00700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 0x7fd3741a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.398+0000 7fd37a800700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 0x7fd3741a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 -- 172.21.15.16:0/2729317224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 msgr2=0x7fd3741a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 0x7fd3741a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 -- 172.21.15.16:0/2729317224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 msgr2=0x7fd3741a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 0x7fd3741a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 -- 172.21.15.16:0/2729317224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741a5f40 con 0x7fd3741039c0 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd37a800700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 0x7fd3741a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.399+0000 7fd379400700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c9b0 tx=0x7fd36400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.400+0000 7fd36ea00700 1 -- 172.21.15.16:0/2729317224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364007b00 con 0x7fd3741039c0 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.400+0000 7fd36ea00700 1 -- 172.21.15.16:0/2729317224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd364007ca0 con 0x7fd3741039c0 2024-05-27T07:02:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.400+0000 7fd36ea00700 1 -- 172.21.15.16:0/2729317224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364018660 con 0x7fd3741039c0 2024-05-27T07:02:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.400+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741a61d0 con 0x7fd3741039c0 2024-05-27T07:02:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.400+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a66c0 con 0x7fd3741039c0 2024-05-27T07:02:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.406+0000 7fd38b3fdb80 1 Processor -- start 2024-05-27T07:02:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 -- start start 2024-05-27T07:02:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd36ea00700 1 -- 172.21.15.16:0/2729317224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd364018800 con 0x7fd3741039c0 2024-05-27T07:02:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643655f5410 0x5643655ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564365523e40 con 0x5643655f5820 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5643655139e0 con 0x564365476e40 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.407+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564365517a20 con 0x5643655f5410 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd37a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52560/0 (socket says 172.21.15.16:52560) 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd37a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:40630/0 (socket says 172.21.15.16:40630) 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd379400700 1 -- 172.21.15.16:0/1916289080 learned_addr learned my addr 172.21.15.16:0/1916289080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.408+0000 7fd379e00700 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643655f5410 0x5643655ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.409+0000 7fd37a800700 1 -- 172.21.15.16:0/1916289080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643655f5410 msgr2=0x5643655ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.409+0000 7fd37a800700 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643655f5410 0x5643655ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.409+0000 7fd37a800700 1 -- 172.21.15.16:0/1916289080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 msgr2=0x564365602020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd37a800700 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd37a800700 1 -- 172.21.15.16:0/1916289080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643654dc420 con 0x564365476e40 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd36ea00700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 0x7fd35c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd379400700 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:02:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd36ea00700 1 -- 172.21.15.16:0/2729317224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3640e2950 con 0x7fd3741039c0 2024-05-27T07:02:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.410+0000 7fd379e00700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 0x7fd35c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.411+0000 7fd37a800700 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b830 tx=0x7fd370009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4ca0cb586ff43d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.411+0000 7fd363400700 1 -- 172.21.15.16:0/1916289080 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370021070 con 0x564365476e40 2024-05-27T07:02:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.411+0000 7fd379e00700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 0x7fd35c0b4f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5643655f6430 tx=0x7fd3680073d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.411+0000 7fd363400700 1 -- 172.21.15.16:0/1916289080 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3700093d0 con 0x564365476e40 2024-05-27T07:02:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.411+0000 7fd363400700 1 -- 172.21.15.16:0/1916289080 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370005040 con 0x564365476e40 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.412+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/1916289080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 msgr2=0x5643655f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.412+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b830 tx=0x7fd370009d30 comp rx=0 tx=0).stop 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/1916289080 shutdown_connections 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5820 0x564365602020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564365476e40 0x5643655f4ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/1916289080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643655f5410 0x5643655ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/1916289080 >> 172.21.15.16:0/1916289080 conn(0x5643654d6fc0 msgr2=0x5643655f0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/1916289080 shutdown_connections 2024-05-27T07:02:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.413+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/1916289080 wait complete. 2024-05-27T07:02:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.414+0000 7fd38b3fdb80 1 -- start start 2024-05-27T07:02:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.414+0000 7fd38b3fdb80 1 Processor -- start 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564365419fa0 con 0x5643655f5410 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5643654b4760 con 0x5643655f5820 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564365602930 con 0x564365476e40 2024-05-27T07:02:39.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd38b3fdb80 10 client.0 _ll_register_callbacks cb 0x5643654bdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:39.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd379e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd379e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:51136/0 (socket says 172.21.15.16:51136) 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd379e00700 1 -- 172.21.15.16:0/2579156133 learned_addr learned my addr 172.21.15.16:0/2579156133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.415+0000 7fd37a800700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379400700 1 -- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 msgr2=0x5643655ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379400700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379400700 1 -- 172.21.15.16:0/2579156133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 msgr2=0x5643656000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379400700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379400700 1 -- 172.21.15.16:0/2579156133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643654638d0 con 0x5643655f5410 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.416+0000 7fd379e00700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.417+0000 7fd379400700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655ffba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd364000c00 tx=0x7fd3640e0260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3a2055dd97234 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.417+0000 7fd357e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.417+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3640a9190 con 0x5643655f5410 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.417+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3640b2210 con 0x5643655f5410 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.417+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3700093d0 con 0x5643655f5410 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.418+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3640a8fb0 con 0x5643655f5410 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.418+0000 7fd38b3fdb80 10 client.15684 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:39.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.418+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643654e4060 con 0x5643655f5410 2024-05-27T07:02:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.418+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3700093d0 con 0x5643655f5410 2024-05-27T07:02:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd3640a8aa0 con 0x5643655f5410 2024-05-27T07:02:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 10 client.15684 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:02:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 20 client.15684 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436560f930 0x564365611d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 4 client.15684 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.419+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564365525d90 con 0x56436560f930 2024-05-27T07:02:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.420+0000 7fd379e00700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436560f930 0x564365611d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.421+0000 7fd379e00700 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436560f930 0x564365611d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=269471011cbfec0 server_cookie=ea01dffd1a2cf84d in_seq=0 out_seq=0 2024-05-27T07:02:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.421+0000 7fd357e00700 10 client.15684 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.423+0000 7fd357e00700 1 -- 172.21.15.16:0/2579156133 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564365525d90 con 0x56436560f930 2024-05-27T07:02:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.423+0000 7fd357e00700 10 client.15684 handle_command_reply: tid=0 2024-05-27T07:02:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 client.15684 shutdown 2024-05-27T07:02:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 msgr2=0x5643655ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655ffba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd364000c00 tx=0x7fd3640e0260 comp rx=0 tx=0).stop 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 shutdown_connections 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436560f930 0x564365611d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x5643655ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/2579156133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655ffba0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 >> 172.21.15.16:0/2579156133 conn(0x5643654d6fc0 msgr2=0x5643655efff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.424+0000 7fd357e00700 0 client.15684 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.425+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 shutdown_connections 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.425+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/2579156133 wait complete. 2024-05-27T07:02:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.425+0000 7fd38b3fdb80 20 client.15684 trim_cache size 0 max 16384 2024-05-27T07:02:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.456+0000 7fd38b3fdb80 1 Processor -- start 2024-05-27T07:02:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.456+0000 7fd38b3fdb80 1 -- start start 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.456+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365601b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.456+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643655ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b67c0 con 0x5643655f5410 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b8300 con 0x5643655f5820 2024-05-27T07:02:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b6e90 con 0x564365476e40 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd379400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52590/0 (socket says 172.21.15.16:52590) 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.457+0000 7fd379400700 1 -- 172.21.15.16:0/619090459 learned_addr learned my addr 172.21.15.16:0/619090459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379e00700 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365601b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd37a800700 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643655ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 -- 172.21.15.16:0/619090459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 msgr2=0x564365601b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365601b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 -- 172.21.15.16:0/619090459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 msgr2=0x5643655ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643655ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 -- 172.21.15.16:0/619090459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643655eb1d0 con 0x5643655f5410 2024-05-27T07:02:39.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.458+0000 7fd379400700 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd36400afd0 tx=0x7fd3640e2c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f24efcae786bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.459+0000 7fd362a00700 1 -- 172.21.15.16:0/619090459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364007e50 con 0x5643655f5410 2024-05-27T07:02:39.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.459+0000 7fd362a00700 1 -- 172.21.15.16:0/619090459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3640a9350 con 0x5643655f5410 2024-05-27T07:02:39.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.459+0000 7fd362a00700 1 -- 172.21.15.16:0/619090459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3640a8aa0 con 0x5643655f5410 2024-05-27T07:02:39.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/619090459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 msgr2=0x5643655fae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd36400afd0 tx=0x7fd3640e2c10 comp rx=0 tx=0).stop 2024-05-27T07:02:39.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/619090459 shutdown_connections 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643655ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365601b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/619090459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643655f5410 0x5643655fae30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/619090459 >> 172.21.15.16:0/619090459 conn(0x5643654d6fc0 msgr2=0x5643655ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/619090459 shutdown_connections 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.460+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/619090459 wait complete. 2024-05-27T07:02:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.461+0000 7fd38b3fdb80 1 -- start start 2024-05-27T07:02:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 Processor -- start 2024-05-27T07:02:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365608270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b7c30 con 0x564365603b10 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b7560 con 0x5643655f5820 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.462+0000 7fd38b3fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b4760 con 0x564365476e40 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd38b3fdb80 10 client.0 _ll_register_callbacks cb 0x5643654bdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd379400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd37a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd37a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52600/0 (socket says 172.21.15.16:52600) 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd37a800700 1 -- 172.21.15.16:0/492536898 learned_addr learned my addr 172.21.15.16:0/492536898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd379400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:40640/0 (socket says 172.21.15.16:40640) 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.463+0000 7fd379e00700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365608270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 -- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 msgr2=0x564365608270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365608270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 -- 172.21.15.16:0/492536898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 msgr2=0x5643656035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 -- 172.21.15.16:0/492536898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643655f2350 con 0x564365603b10 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd37a800700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd370009fd0 tx=0x7fd370004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc29421a20e000c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.464+0000 7fd357e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.465+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37002b070 con 0x564365603b10 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.465+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3700077c0 con 0x564365603b10 2024-05-27T07:02:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.465+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3640a9350 con 0x564365603b10 2024-05-27T07:02:39.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.466+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd370021040 con 0x564365603b10 2024-05-27T07:02:39.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.466+0000 7fd38b3fdb80 10 client.15690 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:39.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.466+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56436567a460 con 0x564365603b10 2024-05-27T07:02:39.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.466+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3640a9350 con 0x564365603b10 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd370011070 con 0x564365603b10 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 10 client.15690 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 20 client.15690 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436567a690 0x56436567cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 4 client.15690 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.467+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56436551f570 con 0x56436567a690 2024-05-27T07:02:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.468+0000 7fd379e00700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436567a690 0x56436567cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.469+0000 7fd379e00700 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436567a690 0x56436567cab0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=adc7313ca9c8f06e server_cookie=20b342ce2f015e63 in_seq=0 out_seq=0 2024-05-27T07:02:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.469+0000 7fd357e00700 10 client.15690 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd357e00700 1 -- 172.21.15.16:0/492536898 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56436551f570 con 0x56436567a690 2024-05-27T07:02:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd357e00700 10 client.15690 handle_command_reply: tid=0 2024-05-27T07:02:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd38b3fdb80 1 client.15690 shutdown 2024-05-27T07:02:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 msgr2=0x564365603f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd370009fd0 tx=0x7fd370004c80 comp rx=0 tx=0).stop 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 shutdown_connections 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.470+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x56436567a690 0x56436567cab0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd357e00700 0 client.15690 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564365603b10 0x564365603f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564365476e40 0x564365608270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 --2- 172.21.15.16:0/492536898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5643655f5820 0x5643656035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 >> 172.21.15.16:0/492536898 conn(0x5643654d6fc0 msgr2=0x5643655ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 shutdown_connections 2024-05-27T07:02:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.471+0000 7fd38b3fdb80 1 -- 172.21.15.16:0/492536898 wait complete. 2024-05-27T07:02:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.472+0000 7fd38b3fdb80 20 client.15690 trim_cache size 0 max 16384 2024-05-27T07:02:39.463 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.472+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 msgr2=0x7fd35c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.472+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 0x7fd35c0b4f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5643655f6430 tx=0x7fd3680073d0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.472+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 msgr2=0x7fd3741a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.472+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c9b0 tx=0x7fd36400ccc0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 shutdown_connections 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd35c0b2a70 0x7fd35c0b4f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3741084c0 0x7fd3741a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd374103070 0x7fd3741a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 --2- 172.21.15.16:0/2729317224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3741039c0 0x7fd3741a16c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.473+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 >> 172.21.15.16:0/2729317224 conn(0x7fd374078ab0 msgr2=0x7fd374105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.474+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 shutdown_connections 2024-05-27T07:02:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:39.474+0000 7fd37b200700 1 -- 172.21.15.16:0/2729317224 wait complete. 2024-05-27T07:02:39.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:39 smithi016 ceph-mon[31415]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T07:02:39.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:39 smithi069 ceph-mon[35276]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T07:02:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:39 smithi164 ceph-mon[35327]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T07:02:40.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:02:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:41 smithi016 ceph-mon[31415]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-27T07:02:41.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:41 smithi069 ceph-mon[35276]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-27T07:02:42.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:41 smithi164 ceph-mon[35327]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 -- 172.21.15.16:0/2508705705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330010e6f0 msgr2=0x7f3300110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 --2- 172.21.15.16:0/2508705705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330010e6f0 0x7f3300110bd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b0a0 tx=0x7f32f800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 -- 172.21.15.16:0/2508705705 shutdown_connections 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 --2- 172.21.15.16:0/2508705705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330010e6f0 0x7f3300110bd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 --2- 172.21.15.16:0/2508705705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105280 0x7f3300105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 --2- 172.21.15.16:0/2508705705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3300105bd0 0x7f330010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.651+0000 7f3307000700 1 -- 172.21.15.16:0/2508705705 >> 172.21.15.16:0/2508705705 conn(0x7f3300078ab0 msgr2=0x7f3300078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.652+0000 7f3307000700 1 -- 172.21.15.16:0/2508705705 shutdown_connections 2024-05-27T07:02:43.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.652+0000 7f3307000700 1 -- 172.21.15.16:0/2508705705 wait complete. 2024-05-27T07:02:43.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.653+0000 7f3307000700 1 Processor -- start 2024-05-27T07:02:43.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.653+0000 7f3307000700 1 -- start start 2024-05-27T07:02:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105bd0 0x7f33001a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f330010e6f0 0x7f330019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3300114150 con 0x7f3300105280 2024-05-27T07:02:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3300113fd0 con 0x7f3300105bd0 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3307000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33001142d0 con 0x7f330010e6f0 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3305c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3305c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44142/0 (socket says 172.21.15.16:44142) 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.654+0000 7f3305c00700 1 -- 172.21.15.16:0/397205336 learned_addr learned my addr 172.21.15.16:0/397205336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305200700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105bd0 0x7f33001a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3306600700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f330010e6f0 0x7f330019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305c00700 1 -- 172.21.15.16:0/397205336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f330010e6f0 msgr2=0x7f330019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305c00700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f330010e6f0 0x7f330019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305c00700 1 -- 172.21.15.16:0/397205336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105bd0 msgr2=0x7f33001a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305c00700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105bd0 0x7f33001a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.655+0000 7f3305c00700 1 -- 172.21.15.16:0/397205336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330019afc0 con 0x7f3300105280 2024-05-27T07:02:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.656+0000 7f3305c00700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32f4011d60 tx=0x7f32f4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.656+0000 7f32fe000700 1 -- 172.21.15.16:0/397205336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4015020 con 0x7f3300105280 2024-05-27T07:02:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.657+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330019b250 con 0x7f3300105280 2024-05-27T07:02:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.657+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3300102ea0 con 0x7f3300105280 2024-05-27T07:02:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.657+0000 7f32fe000700 1 -- 172.21.15.16:0/397205336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32f4011e50 con 0x7f3300105280 2024-05-27T07:02:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.657+0000 7f32fe000700 1 -- 172.21.15.16:0/397205336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f400fbf0 con 0x7f3300105280 2024-05-27T07:02:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.664+0000 7f32fe000700 1 -- 172.21.15.16:0/397205336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f32f4018020 con 0x7f3300105280 2024-05-27T07:02:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.664+0000 7f33170a4b80 1 Processor -- start 2024-05-27T07:02:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.664+0000 7f33170a4b80 1 -- start start 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fcba7820 0x55e9fcbb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9fcad5e40 con 0x55e9fca28e40 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9fcac59e0 con 0x55e9fcba7410 2024-05-27T07:02:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.665+0000 7f33170a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9fcac9a20 con 0x55e9fcba7820 2024-05-27T07:02:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f3306600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f3306600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44148/0 (socket says 172.21.15.16:44148) 2024-05-27T07:02:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f3306600700 1 -- 172.21.15.16:0/95145729 learned_addr learned my addr 172.21.15.16:0/95145729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f3305200700 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fcba7820 0x55e9fcbb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f32fe000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 0x7f32e80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.666+0000 7f32fe000700 1 -- 172.21.15.16:0/397205336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32f40e26a0 con 0x7f3300105280 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 -- 172.21.15.16:0/95145729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fcba7820 msgr2=0x55e9fcbb4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fcba7820 0x55e9fcbb4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 -- 172.21.15.16:0/95145729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 msgr2=0x55e9fcbb1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 -- 172.21.15.16:0/95145729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fca8e420 con 0x55e9fca28e40 2024-05-27T07:02:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.667+0000 7f3306600700 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f32f8015000 tx=0x7f32f8020ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7361bf11df092fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.668+0000 7f3306600700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 0x7f32e80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.668+0000 7f32f2a00700 1 -- 172.21.15.16:0/95145729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80092d0 con 0x55e9fca28e40 2024-05-27T07:02:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.668+0000 7f32f2a00700 1 -- 172.21.15.16:0/95145729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32f8009470 con 0x55e9fca28e40 2024-05-27T07:02:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.668+0000 7f32f2a00700 1 -- 172.21.15.16:0/95145729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80052a0 con 0x55e9fca28e40 2024-05-27T07:02:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.669+0000 7f33170a4b80 1 -- 172.21.15.16:0/95145729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 msgr2=0x55e9fcba6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.669+0000 7f3306600700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 0x7f32e80b4f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b040 tx=0x7f32f8007e60 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.669+0000 7f33170a4b80 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f32f8015000 tx=0x7f32f8020ac0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 -- 172.21.15.16:0/95145729 shutdown_connections 2024-05-27T07:02:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fcba7820 0x55e9fcbb4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fca28e40 0x55e9fcba6ed0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 --2- 172.21.15.16:0/95145729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 -- 172.21.15.16:0/95145729 >> 172.21.15.16:0/95145729 conn(0x55e9fca88fc0 msgr2=0x55e9fcba2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 -- 172.21.15.16:0/95145729 shutdown_connections 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 -- 172.21.15.16:0/95145729 wait complete. 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.670+0000 7f33170a4b80 1 -- start start 2024-05-27T07:02:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.671+0000 7f33170a4b80 1 Processor -- start 2024-05-27T07:02:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9fc9cbfa0 con 0x55e9fcba7820 2024-05-27T07:02:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9fca66760 con 0x55e9fcba7410 2024-05-27T07:02:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9fcbb4920 con 0x55e9fca28e40 2024-05-27T07:02:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.672+0000 7f33170a4b80 10 client.0 _ll_register_callbacks cb 0x55e9fca6fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47752/0 (socket says 172.21.15.16:47752) 2024-05-27T07:02:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3305c00700 1 -- 172.21.15.16:0/3517689481 learned_addr learned my addr 172.21.15.16:0/3517689481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44158/0 (socket says 172.21.15.16:44158) 2024-05-27T07:02:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3305200700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 -- 172.21.15.16:0/3517689481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 msgr2=0x55e9fcbb1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 -- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 msgr2=0x55e9fcbb19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.673+0000 7f3306600700 1 -- 172.21.15.16:0/3517689481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fca158d0 con 0x55e9fcba7820 2024-05-27T07:02:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.674+0000 7f3306600700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f32f800d010 tx=0x7f32f802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495df118c7888c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.674+0000 7f32e7400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.674+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8043060 con 0x55e9fcba7820 2024-05-27T07:02:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.674+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f8009470 con 0x55e9fcba7820 2024-05-27T07:02:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.675+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32f8029e00 con 0x55e9fcba7820 2024-05-27T07:02:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.675+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32f8005a60 con 0x55e9fcba7820 2024-05-27T07:02:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.675+0000 7f33170a4b80 10 client.15702 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.675+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e9fca96060 con 0x55e9fcba7820 2024-05-27T07:02:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.675+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32f8009470 con 0x55e9fcba7820 2024-05-27T07:02:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f32f8011070 con 0x55e9fcba7820 2024-05-27T07:02:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 10 client.15702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:02:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 20 client.15702 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcbc1560 0x55e9fcbc3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 4 client.15702 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.676+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9fcad7d90 con 0x55e9fcbc1560 2024-05-27T07:02:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.677+0000 7f3305c00700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcbc1560 0x55e9fcbc3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.678+0000 7f3305c00700 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcbc1560 0x55e9fcbc3980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d4483e1c54b8779 server_cookie=2e31f530eaecb9b in_seq=0 out_seq=0 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.678+0000 7f32e7400700 10 client.15702 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.680+0000 7f32e7400700 1 -- 172.21.15.16:0/3517689481 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9fcad7d90 con 0x55e9fcbc1560 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.680+0000 7f32e7400700 10 client.15702 handle_command_reply: tid=0 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.680+0000 7f33170a4b80 1 client.15702 shutdown 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 msgr2=0x55e9fcbb1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f32f800d010 tx=0x7f32f802b000 comp rx=0 tx=0).stop 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 shutdown_connections 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbb1f10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcbc1560 0x55e9fcbc3980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 --2- 172.21.15.16:0/3517689481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbb19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 >> 172.21.15.16:0/3517689481 conn(0x55e9fca88fc0 msgr2=0x55e9fcba1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f32e7400700 0 client.15702 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 shutdown_connections 2024-05-27T07:02:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.681+0000 7f33170a4b80 1 -- 172.21.15.16:0/3517689481 wait complete. 2024-05-27T07:02:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.682+0000 7f33170a4b80 20 client.15702 trim_cache size 0 max 16384 2024-05-27T07:02:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.712+0000 7f33170a4b80 1 Processor -- start 2024-05-27T07:02:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.712+0000 7f33170a4b80 1 -- start start 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7820 0x55e9fcbbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000edd0 con 0x55e9fcba7410 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0010910 con 0x55e9fcba7820 2024-05-27T07:02:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f33170a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000f4a0 con 0x55e9fca28e40 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f3305200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47762/0 (socket says 172.21.15.16:47762) 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f3305c00700 1 -- 172.21.15.16:0/728006377 learned_addr learned my addr 172.21.15.16:0/728006377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.713+0000 7f3305200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44162/0 (socket says 172.21.15.16:44162) 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3306600700 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7820 0x55e9fcbbdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 -- 172.21.15.16:0/728006377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 msgr2=0x55e9fcbb07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 -- 172.21.15.16:0/728006377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7820 msgr2=0x55e9fcbbdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7820 0x55e9fcbbdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 -- 172.21.15.16:0/728006377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fcba4350 con 0x55e9fcba7410 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305c00700 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.714+0000 7f3305200700 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55e9fcb9c510 tx=0x7f32ec011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea809dd499f51f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.715+0000 7f32f2000700 1 -- 172.21.15.16:0/728006377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec011d50 con 0x55e9fcba7410 2024-05-27T07:02:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.715+0000 7f32f2000700 1 -- 172.21.15.16:0/728006377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32ec0044c0 con 0x55e9fcba7410 2024-05-27T07:02:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.715+0000 7f32f2000700 1 -- 172.21.15.16:0/728006377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec005280 con 0x55e9fcba7410 2024-05-27T07:02:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.716+0000 7f33170a4b80 1 -- 172.21.15.16:0/728006377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 msgr2=0x55e9fcbb3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.716+0000 7f33170a4b80 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55e9fcb9c510 tx=0x7f32ec011720 comp rx=0 tx=0).stop 2024-05-27T07:02:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 -- 172.21.15.16:0/728006377 shutdown_connections 2024-05-27T07:02:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7820 0x55e9fcbbdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbb07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 --2- 172.21.15.16:0/728006377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7410 0x55e9fcbb3900 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 -- 172.21.15.16:0/728006377 >> 172.21.15.16:0/728006377 conn(0x55e9fca88fc0 msgr2=0x55e9fcbb1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 -- 172.21.15.16:0/728006377 shutdown_connections 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.717+0000 7f33170a4b80 1 -- 172.21.15.16:0/728006377 wait complete. 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.718+0000 7f33170a4b80 1 -- start start 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.718+0000 7f33170a4b80 1 Processor -- start 2024-05-27T07:02:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbc6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbc4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbc4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0010240 con 0x55e9fcba7820 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000fb70 con 0x55e9fcba7410 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000cd10 con 0x55e9fca28e40 2024-05-27T07:02:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f33170a4b80 10 client.0 _ll_register_callbacks cb 0x55e9fca6fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbc6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f3305c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbc6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2024-05-27T07:02:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.719+0000 7f3305c00700 1 -- 172.21.15.16:0/494468085 learned_addr learned my addr 172.21.15.16:0/494468085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbc4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3305200700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbc4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 -- 172.21.15.16:0/494468085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 msgr2=0x55e9fcbc6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbc6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 -- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 msgr2=0x55e9fcbc4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbc4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3306600700 1 -- 172.21.15.16:0/494468085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fcbc4bb0 con 0x55e9fcba7820 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.720+0000 7f3305200700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbc4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.721+0000 7f3306600700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbc4670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009900 tx=0x7f32f8029ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d64683e805ecf6d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.721+0000 7f32e7400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.721+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8007cc0 con 0x55e9fcba7820 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.721+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32f8029e00 con 0x55e9fcba7820 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.721+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec0044c0 con 0x55e9fcba7820 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.722+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32f8056050 con 0x55e9fcba7820 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.722+0000 7f33170a4b80 10 client.15708 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.722+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e9fcc2b9d0 con 0x55e9fcba7820 2024-05-27T07:02:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.722+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32ec0044c0 con 0x55e9fcba7820 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f32f8011070 con 0x55e9fcba7820 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 10 client.15708 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 20 client.15708 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcc2c0d0 0x55e9fcc2e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 4 client.15708 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9fcad1570 con 0x55e9fcc2c0d0 2024-05-27T07:02:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.723+0000 7f3305c00700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcc2c0d0 0x55e9fcc2e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.724+0000 7f3305c00700 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcc2c0d0 0x55e9fcc2e4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e929c1efc45b108 server_cookie=4270a3e0d86c8b24 in_seq=0 out_seq=0 2024-05-27T07:02:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.724+0000 7f32e7400700 10 client.15708 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f32e7400700 1 -- 172.21.15.16:0/494468085 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9fcad1570 con 0x55e9fcc2c0d0 2024-05-27T07:02:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f32e7400700 10 client.15708 handle_command_reply: tid=0 2024-05-27T07:02:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 client.15708 shutdown 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 msgr2=0x55e9fcbc4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbc4670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009900 tx=0x7f32f8029ae0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 shutdown_connections 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9fcba7820 0x55e9fcbc4670 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9fca28e40 0x55e9fcbc6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9fcc2c0d0 0x55e9fcc2e4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 --2- 172.21.15.16:0/494468085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9fcba7410 0x55e9fcbc4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.725+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 >> 172.21.15.16:0/494468085 conn(0x55e9fca88fc0 msgr2=0x55e9fcbb1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.726+0000 7f32e7400700 0 client.15708 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:02:43.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.726+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 shutdown_connections 2024-05-27T07:02:43.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.726+0000 7f33170a4b80 1 -- 172.21.15.16:0/494468085 wait complete. 2024-05-27T07:02:43.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.726+0000 7f33170a4b80 20 client.15708 trim_cache size 0 max 16384 2024-05-27T07:02:43.718 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 msgr2=0x7f32e80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 0x7f32e80b4f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b040 tx=0x7f32f8007e60 comp rx=0 tx=0).stop 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 msgr2=0x7f33001a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32f4011d60 tx=0x7f32f4009520 comp rx=0 tx=0).stop 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 shutdown_connections 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32e80b2a70 0x7f32e80b4f80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f330010e6f0 0x7f330019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300105280 0x7f33001a06f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.727+0000 7f3307000700 1 --2- 172.21.15.16:0/397205336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3300105bd0 0x7f33001a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:43.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.728+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 >> 172.21.15.16:0/397205336 conn(0x7f3300078ab0 msgr2=0x7f33000fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:43.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.728+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 shutdown_connections 2024-05-27T07:02:43.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:43.728+0000 7f3307000700 1 -- 172.21.15.16:0/397205336 wait complete. 2024-05-27T07:02:43.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:43 smithi016 ceph-mon[31415]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:43.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:43 smithi069 ceph-mon[35276]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:43 smithi164 ceph-mon[35327]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-27T07:02:44.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:02:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:45.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:45 smithi016 ceph-mon[31415]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:45.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:45 smithi069 ceph-mon[35276]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:45 smithi164 ceph-mon[35327]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:47.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:47 smithi069 ceph-mon[35276]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 -- 172.21.15.16:0/3607334214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598107e20 msgr2=0x7f3598108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 --2- 172.21.15.16:0/3607334214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598107e20 0x7f3598108230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f35840099c0 tx=0x7f3584009cd0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 -- 172.21.15.16:0/3607334214 shutdown_connections 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 --2- 172.21.15.16:0/3607334214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598108770 0x7f35981100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 --2- 172.21.15.16:0/3607334214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598103540 0x7f3598103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 --2- 172.21.15.16:0/3607334214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598107e20 0x7f3598108230 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 -- 172.21.15.16:0/3607334214 >> 172.21.15.16:0/3607334214 conn(0x7f3598078df0 msgr2=0x7f3598079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 -- 172.21.15.16:0/3607334214 shutdown_connections 2024-05-27T07:02:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.927+0000 7f359ec00700 1 -- 172.21.15.16:0/3607334214 wait complete. 2024-05-27T07:02:47.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:47 smithi016 ceph-mon[31415]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.929+0000 7f359ec00700 1 Processor -- start 2024-05-27T07:02:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.929+0000 7f359ec00700 1 -- start start 2024-05-27T07:02:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 0x7f359819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 0x7f359819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598108770 0x7f359819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3598113280 con 0x7f3598108770 2024-05-27T07:02:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3598113100 con 0x7f3598107e20 2024-05-27T07:02:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f359ec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3598113400 con 0x7f3598103540 2024-05-27T07:02:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f3597e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 0x7f359819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f3597400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 0x7f359819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f3597e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 0x7f359819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47790/0 (socket says 172.21.15.16:47790) 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.930+0000 7f3597e00700 1 -- 172.21.15.16:0/1027750713 learned_addr learned my addr 172.21.15.16:0/1027750713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f359cc00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598108770 0x7f359819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f3597400700 1 -- 172.21.15.16:0/1027750713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 msgr2=0x7f359819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f3597400700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 0x7f359819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f3597400700 1 -- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598108770 msgr2=0x7f359819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f3597400700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598108770 0x7f359819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.931+0000 7f3597400700 1 -- 172.21.15.16:0/1027750713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359819fc60 con 0x7f3598107e20 2024-05-27T07:02:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.932+0000 7f3597400700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 0x7f359819b3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35840098e0 tx=0x7f35840057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.933+0000 7f3594c00700 1 -- 172.21.15.16:0/1027750713 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584027070 con 0x7f3598107e20 2024-05-27T07:02:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.933+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359819fef0 con 0x7f3598107e20 2024-05-27T07:02:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.933+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981a0420 con 0x7f3598107e20 2024-05-27T07:02:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.933+0000 7f3594c00700 1 -- 172.21.15.16:0/1027750713 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358400fc80 con 0x7f3598107e20 2024-05-27T07:02:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.933+0000 7f3594c00700 1 -- 172.21.15.16:0/1027750713 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584022d90 con 0x7f3598107e20 2024-05-27T07:02:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.938+0000 7f35ad7c6b80 1 Processor -- start 2024-05-27T07:02:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.938+0000 7f35ad7c6b80 1 -- start start 2024-05-27T07:02:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.938+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.938+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb01de40 0x5649cb19c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb1a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5649cb0cae40 con 0x5649cb01de40 2024-05-27T07:02:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649cb0ba9e0 con 0x5649cb19ce70 2024-05-27T07:02:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5649cb0bea20 con 0x5649cb1099f0 2024-05-27T07:02:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f359cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f359cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47806/0 (socket says 172.21.15.16:47806) 2024-05-27T07:02:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f359cc00700 1 -- 172.21.15.16:0/3649409163 learned_addr learned my addr 172.21.15.16:0/3649409163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.939+0000 7f3597e00700 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb01de40 0x5649cb19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f359cc00700 1 -- 172.21.15.16:0/3649409163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 msgr2=0x5649cb1a9110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f3594c00700 1 -- 172.21.15.16:0/1027750713 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f358402a070 con 0x7f3598107e20 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f359cc00700 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb1a9110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f359cc00700 1 -- 172.21.15.16:0/3649409163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb01de40 msgr2=0x5649cb19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f359cc00700 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb01de40 0x5649cb19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.940+0000 7f359cc00700 1 -- 172.21.15.16:0/3649409163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cb083420 con 0x5649cb1099f0 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.941+0000 7f359cc00700 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3590007aa0 tx=0x7f3590012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea844b157cb007f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.941+0000 7f358d600700 1 -- 172.21.15.16:0/3649409163 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590004bb0 con 0x5649cb1099f0 2024-05-27T07:02:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.942+0000 7f358d600700 1 -- 172.21.15.16:0/3649409163 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3590004d50 con 0x5649cb1099f0 2024-05-27T07:02:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.942+0000 7f358d600700 1 -- 172.21.15.16:0/3649409163 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590017670 con 0x5649cb1099f0 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.942+0000 7f3594c00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 0x7f357c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.942+0000 7f3594c00700 1 -- 172.21.15.16:0/1027750713 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f35840e36e0 con 0x7f3598107e20 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.943+0000 7f3597e00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 0x7f357c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/3649409163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 msgr2=0x5649cb19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3590007aa0 tx=0x7f3590012b30 comp rx=0 tx=0).stop 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/3649409163 shutdown_connections 2024-05-27T07:02:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb1a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb19c330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/3649409163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb01de40 0x5649cb19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.944+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/3649409163 >> 172.21.15.16:0/3649409163 conn(0x5649cb0b7390 msgr2=0x5649cb1984c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.945+0000 7f3597e00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 0x7f357c0b4f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5649cb199790 tx=0x7f358800b500 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.945+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/3649409163 shutdown_connections 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.945+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/3649409163 wait complete. 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.946+0000 7f35ad7c6b80 1 -- start start 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.946+0000 7f35ad7c6b80 1 Processor -- start 2024-05-27T07:02:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 0x5649cb1a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5649cafc0fa0 con 0x5649cb19ce70 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649cb05b760 con 0x5649cb01de40 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5649cb1a9b20 con 0x5649cb1099f0 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f35ad7c6b80 10 client.0 _ll_register_callbacks cb 0x5649cb064c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.947+0000 7f359cc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 0x5649cb1a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47814/0 (socket says 172.21.15.16:47814) 2024-05-27T07:02:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 0x5649cb1a66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34096/0 (socket says 172.21.15.16:34096) 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 -- 172.21.15.16:0/1925196757 learned_addr learned my addr 172.21.15.16:0/1925196757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 -- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 msgr2=0x5649cb1a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 0x5649cb1a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 -- 172.21.15.16:0/1925196757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 msgr2=0x5649cb1a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f3597400700 1 -- 172.21.15.16:0/1925196757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cb00a8d0 con 0x5649cb1099f0 2024-05-27T07:02:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.948+0000 7f359cc00700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.949+0000 7f3597400700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3584005f50 tx=0x7f35840aaba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3008dbb86b3b618 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.949+0000 7f3582000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.949+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840a9dd0 con 0x5649cb1099f0 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.949+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f35840e3280 con 0x5649cb1099f0 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.950+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3590004d50 con 0x5649cb1099f0 2024-05-27T07:02:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.950+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f35840aa170 con 0x5649cb1099f0 2024-05-27T07:02:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.950+0000 7f35ad7c6b80 10 client.25417 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.950+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649cb08b060 con 0x5649cb1099f0 2024-05-27T07:02:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.951+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3590004d50 con 0x5649cb1099f0 2024-05-27T07:02:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f35840afba0 con 0x5649cb1099f0 2024-05-27T07:02:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 10 client.25417 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 10 client.25417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:02:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 20 client.25417 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb1b6aa0 0x5649cb1b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 4 client.25417 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.952+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649cb0ccd90 con 0x5649cb1b6aa0 2024-05-27T07:02:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.953+0000 7f3597e00700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb1b6aa0 0x5649cb1b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.954+0000 7f3597e00700 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb1b6aa0 0x5649cb1b8ec0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df19a1299ae4897a server_cookie=724cb60e4b5b605 in_seq=0 out_seq=0 2024-05-27T07:02:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.954+0000 7f3582000700 10 client.25417 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.956+0000 7f3582000700 1 -- 172.21.15.16:0/1925196757 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5649cb0ccd90 con 0x5649cb1b6aa0 2024-05-27T07:02:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.956+0000 7f3582000700 10 client.25417 handle_command_reply: tid=0 2024-05-27T07:02:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.956+0000 7f35ad7c6b80 1 client.25417 shutdown 2024-05-27T07:02:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 msgr2=0x5649cb1a6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3584005f50 tx=0x7f35840aaba0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 shutdown_connections 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb1b6aa0 0x5649cb1b8ec0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb01de40 0x5649cb1a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1925196757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb1099f0 0x5649cb1a6c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 >> 172.21.15.16:0/1925196757 conn(0x5649cb0b7390 msgr2=0x5649cb19c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f3582000700 0 client.25417 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 shutdown_connections 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1925196757 wait complete. 2024-05-27T07:02:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.957+0000 7f35ad7c6b80 20 client.25417 trim_cache size 0 max 16384 2024-05-27T07:02:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.989+0000 7f35ad7c6b80 1 Processor -- start 2024-05-27T07:02:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.989+0000 7f35ad7c6b80 1 -- start start 2024-05-27T07:02:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb19c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f357800ec80 con 0x5649cb1099f0 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35780107c0 con 0x5649cb19ce70 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f357800f350 con 0x5649cb01de40 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.990+0000 7f3597400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44226/0 (socket says 172.21.15.16:44226) 2024-05-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 -- 172.21.15.16:0/1277757342 learned_addr learned my addr 172.21.15.16:0/1277757342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f359cc00700 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 -- 172.21.15.16:0/1277757342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 msgr2=0x5649cb1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 -- 172.21.15.16:0/1277757342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 msgr2=0x5649cb19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.991+0000 7f3597400700 1 -- 172.21.15.16:0/1277757342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cb1a74b0 con 0x5649cb1099f0 2024-05-27T07:02:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.992+0000 7f3597400700 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f35840aacc0 tx=0x7f35840a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e03b2287178cca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.992+0000 7f358cc00700 1 -- 172.21.15.16:0/1277757342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840a9dd0 con 0x5649cb1099f0 2024-05-27T07:02:47.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.993+0000 7f358cc00700 1 -- 172.21.15.16:0/1277757342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358406d020 con 0x5649cb1099f0 2024-05-27T07:02:47.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.993+0000 7f358cc00700 1 -- 172.21.15.16:0/1277757342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840aa430 con 0x5649cb1099f0 2024-05-27T07:02:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1277757342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 msgr2=0x5649cb1a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f35840aacc0 tx=0x7f35840a65f0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1277757342 shutdown_connections 2024-05-27T07:02:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb19ce70 0x5649cb19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1277757342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb1099f0 0x5649cb1a1eb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1277757342 >> 172.21.15.16:0/1277757342 conn(0x5649cb0b7390 msgr2=0x5649cb19c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.994+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1277757342 shutdown_connections 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.995+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1277757342 wait complete. 2024-05-27T07:02:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.996+0000 7f35ad7c6b80 1 -- start start 2024-05-27T07:02:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.996+0000 7f35ad7c6b80 1 Processor -- start 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb1a1090 0x5649cb1ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35780100f0 con 0x5649cb19ce70 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f357800fa20 con 0x5649cb1a1090 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f357800cc20 con 0x5649cb01de40 2024-05-27T07:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f35ad7c6b80 10 client.0 _ll_register_callbacks cb 0x5649cb064c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f3597400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f3597e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f3597400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44234/0 (socket says 172.21.15.16:44234) 2024-05-27T07:02:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f3597400700 1 -- 172.21.15.16:0/1669283532 learned_addr learned my addr 172.21.15.16:0/1669283532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.997+0000 7f359cc00700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb1a1090 0x5649cb1ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.998+0000 7f3597400700 1 -- 172.21.15.16:0/1669283532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 msgr2=0x5649cb1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.998+0000 7f3597400700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.998+0000 7f3597400700 1 -- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb1a1090 msgr2=0x5649cb1ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.998+0000 7f3597400700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb1a1090 0x5649cb1ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.998+0000 7f3597400700 1 -- 172.21.15.16:0/1669283532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cb1ae480 con 0x5649cb19ce70 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.999+0000 7f3597400700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f35840a78d0 tx=0x7f35840053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1c713cda2cb822 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.999+0000 7f3582000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.999+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840aa430 con 0x5649cb19ce70 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.999+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f35840a6c50 con 0x5649cb19ce70 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:47.999+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358406d020 con 0x5649cb19ce70 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f35840a9dd0 con 0x5649cb19ce70 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 10 client.15726 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649cb1ae480 con 0x5649cb19ce70 2024-05-27T07:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358406d020 con 0x5649cb19ce70 2024-05-27T07:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f358400f7f0 con 0x5649cb19ce70 2024-05-27T07:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 20 client.15726 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb221530 0x5649cb223950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 4 client.15726 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.000+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649cb0c6570 con 0x5649cb221530 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.001+0000 7f3597e00700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb221530 0x5649cb223950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.002+0000 7f3597e00700 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb221530 0x5649cb223950 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9631919fc7db5ddb server_cookie=feee40671bd76c0d in_seq=0 out_seq=0 2024-05-27T07:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.002+0000 7f3582000700 10 client.15726 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.003+0000 7f3582000700 1 -- 172.21.15.16:0/1669283532 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649cb0c6570 con 0x5649cb221530 2024-05-27T07:02:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.003+0000 7f3582000700 10 client.15726 handle_command_reply: tid=0 2024-05-27T07:02:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.003+0000 7f35ad7c6b80 1 client.15726 shutdown 2024-05-27T07:02:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.003+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 msgr2=0x5649cb1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.003+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f35840a78d0 tx=0x7f35840053c0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 shutdown_connections 2024-05-27T07:02:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x5649cb221530 0x5649cb223950 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649cb1a1090 0x5649cb1ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cb01de40 0x5649cb1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 --2- 172.21.15.16:0/1669283532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649cb19ce70 0x5649cb1a0b50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 >> 172.21.15.16:0/1669283532 conn(0x5649cb0b7390 msgr2=0x5649cb19ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f3582000700 0 client.15726 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 shutdown_connections 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 1 -- 172.21.15.16:0/1669283532 wait complete. 2024-05-27T07:02:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.004+0000 7f35ad7c6b80 20 client.15726 trim_cache size 0 max 16384 2024-05-27T07:02:47.998 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:47.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 msgr2=0x7f357c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 0x7f357c0b4f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5649cb199790 tx=0x7f358800b500 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 msgr2=0x7f359819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 0x7f359819b3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35840098e0 tx=0x7f35840057c0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 shutdown_connections 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f357c0b29f0 0x7f357c0b4f00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3598108770 0x7f359819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3598103540 0x7f359819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.005+0000 7f359ec00700 1 --2- 172.21.15.16:0/1027750713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3598107e20 0x7f359819b3e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.006+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 >> 172.21.15.16:0/1027750713 conn(0x7f3598078df0 msgr2=0x7f3598106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.006+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 shutdown_connections 2024-05-27T07:02:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:48.006+0000 7f359ec00700 1 -- 172.21.15.16:0/1027750713 wait complete. 2024-05-27T07:02:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:47 smithi164 ceph-mon[35327]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:49.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:49.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:49 smithi016 ceph-mon[31415]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:50.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:49 smithi164 ceph-mon[35327]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:50.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:49 smithi069 ceph-mon[35276]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:51.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:51 smithi016 ceph-mon[31415]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 -- 172.21.15.16:0/3653004866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc1043f0 msgr2=0x7f11bc104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 --2- 172.21.15.16:0/3653004866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc1043f0 0x7f11bc104800 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f11b00099c0 tx=0x7f11b0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 -- 172.21.15.16:0/3653004866 shutdown_connections 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 --2- 172.21.15.16:0/3653004866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 0x7f11bc110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 --2- 172.21.15.16:0/3653004866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc1043f0 0x7f11bc104800 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 --2- 172.21.15.16:0/3653004866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc108d20 0x7f11bc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 -- 172.21.15.16:0/3653004866 >> 172.21.15.16:0/3653004866 conn(0x7f11bc079d00 msgr2=0x7f11bc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 -- 172.21.15.16:0/3653004866 shutdown_connections 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.035+0000 7f11c2000700 1 -- 172.21.15.16:0/3653004866 wait complete. 2024-05-27T07:02:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 Processor -- start 2024-05-27T07:02:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 -- start start 2024-05-27T07:02:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc1043f0 0x7f11bc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 0x7f11bc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1141a0 con 0x7f11bc108d20 2024-05-27T07:02:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc114020 con 0x7f11bc1043f0 2024-05-27T07:02:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.037+0000 7f11c2000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc114320 con 0x7f11bc109670 2024-05-27T07:02:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.038+0000 7f11bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.038+0000 7f11bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37184/0 (socket says 172.21.15.16:37184) 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.038+0000 7f11bbe00700 1 -- 172.21.15.16:0/3193605008 learned_addr learned my addr 172.21.15.16:0/3193605008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.038+0000 7f11c0c00700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc1043f0 0x7f11bc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.038+0000 7f11c1600700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 0x7f11bc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 -- 172.21.15.16:0/3193605008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 msgr2=0x7f11bc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 0x7f11bc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 -- 172.21.15.16:0/3193605008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc1043f0 msgr2=0x7f11bc1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc1043f0 0x7f11bc1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 -- 172.21.15.16:0/3193605008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19afc0 con 0x7f11bc108d20 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11bbe00700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c9e0 tx=0x7f11a800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11b9600700 1 -- 172.21.15.16:0/3193605008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8007aa0 con 0x7f11bc108d20 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19b2b0 con 0x7f11bc108d20 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11b9600700 1 -- 172.21.15.16:0/3193605008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11a8007c40 con 0x7f11bc108d20 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.039+0000 7f11b9600700 1 -- 172.21.15.16:0/3193605008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8018400 con 0x7f11bc108d20 2024-05-27T07:02:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.040+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a75f0 con 0x7f11bc108d20 2024-05-27T07:02:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.046+0000 7f11b9600700 1 -- 172.21.15.16:0/3193605008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f11a800f450 con 0x7f11bc108d20 2024-05-27T07:02:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.046+0000 7f11d21bcb80 1 Processor -- start 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.046+0000 7f11d21bcb80 1 -- start start 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c82ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 0x560461c8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83820 0x560461c90020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560461bb1e40 con 0x560461b04e40 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560461ba19e0 con 0x560461c83820 2024-05-27T07:02:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.047+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560461ba5a20 con 0x560461c83410 2024-05-27T07:02:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.048+0000 7f11c1600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c82ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.048+0000 7f11c1600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c82ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37188/0 (socket says 172.21.15.16:37188) 2024-05-27T07:02:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.048+0000 7f11c1600700 1 -- 172.21.15.16:0/1720231687 learned_addr learned my addr 172.21.15.16:0/1720231687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.048+0000 7f11c0c00700 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 0x560461c8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.048+0000 7f11bbe00700 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83820 0x560461c90020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11b9600700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 0x7f11a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11c0c00700 1 -- 172.21.15.16:0/1720231687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83820 msgr2=0x560461c90020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11c0c00700 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83820 0x560461c90020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11c0c00700 1 -- 172.21.15.16:0/1720231687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 msgr2=0x560461c82ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11c0c00700 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c82ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11c0c00700 1 -- 172.21.15.16:0/1720231687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560461b6a420 con 0x560461c83410 2024-05-27T07:02:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.049+0000 7f11b9600700 1 -- 172.21.15.16:0/3193605008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f11a80e2930 con 0x7f11bc108d20 2024-05-27T07:02:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.050+0000 7f11c1600700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 0x7f11a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.050+0000 7f11c0c00700 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 0x560461c8dad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098e0 tx=0x7f11b0004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f5a703a37b6570 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.050+0000 7f11ae000700 1 -- 172.21.15.16:0/1720231687 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0027070 con 0x560461c83410 2024-05-27T07:02:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.051+0000 7f11ae000700 1 -- 172.21.15.16:0/1720231687 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11b000f460 con 0x560461c83410 2024-05-27T07:02:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.051+0000 7f11ae000700 1 -- 172.21.15.16:0/1720231687 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b00226a0 con 0x560461c83410 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 -- 172.21.15.16:0/1720231687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 msgr2=0x560461c8dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11c1600700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 0x7f11a40b4f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x560461c7f690 tx=0x7f11b4009040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 0x560461c8dad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098e0 tx=0x7f11b0004740 comp rx=0 tx=0).stop 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 -- 172.21.15.16:0/1720231687 shutdown_connections 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83820 0x560461c90020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c82ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/1720231687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83410 0x560461c8dad0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 -- 172.21.15.16:0/1720231687 >> 172.21.15.16:0/1720231687 conn(0x560461b64fc0 msgr2=0x560461c7ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 -- 172.21.15.16:0/1720231687 shutdown_connections 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.052+0000 7f11d21bcb80 1 -- 172.21.15.16:0/1720231687 wait complete. 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.053+0000 7f11d21bcb80 1 -- start start 2024-05-27T07:02:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.053+0000 7f11d21bcb80 1 Processor -- start 2024-05-27T07:02:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c8e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560461aa7fa0 con 0x560461b04e40 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560461b42760 con 0x560461c83410 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560461c90930 con 0x560461c83820 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11d21bcb80 10 client.0 _ll_register_callbacks cb 0x560461b4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11c0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11c0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c8d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37196/0 (socket says 172.21.15.16:37196) 2024-05-27T07:02:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.054+0000 7f11c0c00700 1 -- 172.21.15.16:0/3804612424 learned_addr learned my addr 172.21.15.16:0/3804612424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.055+0000 7f11c1600700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c8e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.055+0000 7f11bbe00700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 -- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 msgr2=0x560461c8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 -- 172.21.15.16:0/3804612424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 msgr2=0x560461c8d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c8d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 -- 172.21.15.16:0/3804612424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560461af18d0 con 0x560461c83820 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11bbe00700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11c1600700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c8e0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11b4016e30 tx=0x7f11b4007950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d8efe37d781188 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.056+0000 7f11a2a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.057+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4015e10 con 0x560461c83820 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.057+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b000f460 con 0x560461c83820 2024-05-27T07:02:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.057+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f11b4010040 con 0x560461c83820 2024-05-27T07:02:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.058+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11b402f050 con 0x560461c83820 2024-05-27T07:02:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.058+0000 7f11d21bcb80 10 client.25429 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.058+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560461b72060 con 0x560461c83820 2024-05-27T07:02:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.058+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11b000f460 con 0x560461c83820 2024-05-27T07:02:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11b4022b70 con 0x560461c83820 2024-05-27T07:02:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 10 client.25429 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 10 client.25429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:02:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 20 client.25429 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461ca6180 0x560461ca8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 4 client.25429 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.059+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560461bb3d90 con 0x560461ca6180 2024-05-27T07:02:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.060+0000 7f11c0c00700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461ca6180 0x560461ca8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.061+0000 7f11a2a00700 10 client.25429 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.061+0000 7f11c0c00700 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461ca6180 0x560461ca8620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a54f19b0c3057546 server_cookie=a488f8e926f9e450 in_seq=0 out_seq=0 2024-05-27T07:02:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.063+0000 7f11a2a00700 1 -- 172.21.15.16:0/3804612424 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560461bb3d90 con 0x560461ca6180 2024-05-27T07:02:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.063+0000 7f11a2a00700 10 client.25429 handle_command_reply: tid=0 2024-05-27T07:02:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.063+0000 7f11d21bcb80 1 client.25429 shutdown 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 msgr2=0x560461c8e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c8e0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11b4016e30 tx=0x7f11b4007950 comp rx=0 tx=0).stop 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 shutdown_connections 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461ca6180 0x560461ca8620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c8e0b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c8d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/3804612424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 >> 172.21.15.16:0/3804612424 conn(0x560461b64fc0 msgr2=0x560461c7dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11a2a00700 0 client.25429 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 shutdown_connections 2024-05-27T07:02:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.064+0000 7f11d21bcb80 1 -- 172.21.15.16:0/3804612424 wait complete. 2024-05-27T07:02:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.065+0000 7f11d21bcb80 20 client.25429 trim_cache size 0 max 16384 2024-05-27T07:02:52.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:51 smithi164 ceph-mon[35327]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 Processor -- start 2024-05-27T07:02:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 -- start start 2024-05-27T07:02:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1adfd0 con 0x560461b04e40 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1afb10 con 0x560461c83410 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.096+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ae6a0 con 0x560461c83820 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.097+0000 7f11c0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.097+0000 7f11c0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37210/0 (socket says 172.21.15.16:37210) 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.097+0000 7f11c0c00700 1 -- 172.21.15.16:0/2570388468 learned_addr learned my addr 172.21.15.16:0/2570388468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.097+0000 7f11c1600700 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.097+0000 7f11c0c00700 1 -- 172.21.15.16:0/2570388468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 msgr2=0x560461c877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c0c00700 1 -- 172.21.15.16:0/2570388468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 msgr2=0x560461c8faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c0c00700 1 -- 172.21.15.16:0/2570388468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560461ca3c00 con 0x560461b04e40 2024-05-27T07:02:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c1600700 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.098+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009fd0 tx=0x7f11b000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1cf1de44269c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.099+0000 7f11ad600700 1 -- 172.21.15.16:0/2570388468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0027040 con 0x560461b04e40 2024-05-27T07:02:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.099+0000 7f11ad600700 1 -- 172.21.15.16:0/2570388468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11b0035070 con 0x560461b04e40 2024-05-27T07:02:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.099+0000 7f11ad600700 1 -- 172.21.15.16:0/2570388468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0005480 con 0x560461b04e40 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2570388468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 msgr2=0x560461c7eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009fd0 tx=0x7f11b000bbb0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2570388468 shutdown_connections 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461c83820 0x560461c877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461b04e40 0x560461c7eed0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2570388468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.100+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2570388468 >> 172.21.15.16:0/2570388468 conn(0x560461b64fc0 msgr2=0x560461c7e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.101+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2570388468 shutdown_connections 2024-05-27T07:02:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.101+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2570388468 wait complete. 2024-05-27T07:02:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.102+0000 7f11d21bcb80 1 -- start start 2024-05-27T07:02:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.102+0000 7f11d21bcb80 1 Processor -- start 2024-05-27T07:02:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461b04e40 0x560461c9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c9ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1af440 con 0x560461c83820 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1aed70 con 0x560461c83410 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1abf70 con 0x560461b04e40 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.103+0000 7f11d21bcb80 10 client.0 _ll_register_callbacks cb 0x560461b4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11bbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c9ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37214/0 (socket says 172.21.15.16:37214) 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 -- 172.21.15.16:0/2138691693 learned_addr learned my addr 172.21.15.16:0/2138691693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461b04e40 0x560461c9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 -- 172.21.15.16:0/2138691693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461b04e40 msgr2=0x560461c9e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461b04e40 0x560461c9e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 -- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 msgr2=0x560461c9ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c9ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.104+0000 7f11c1600700 1 -- 172.21.15.16:0/2138691693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560461c8e420 con 0x560461c83820 2024-05-27T07:02:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.105+0000 7f11c1600700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f11b4015b00 tx=0x7f11b4008fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea91a06e53482b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.105+0000 7f11a2a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.105+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4005330 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.105+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f11b4005e70 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.106+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b0035070 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.106+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11b4004a10 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.106+0000 7f11d21bcb80 10 client.15744 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.106+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560461c80350 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.106+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11b0035070 con 0x560461c83820 2024-05-27T07:02:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11b403c020 con 0x560461c83820 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 20 client.15744 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461d10930 0x560461d12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 4 client.15744 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.107+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560461bad570 con 0x560461d10930 2024-05-27T07:02:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.108+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461d10930 0x560461d12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.109+0000 7f11c0c00700 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461d10930 0x560461d12d50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=830c9433d17b835a server_cookie=cd18119419f733a5 in_seq=0 out_seq=0 2024-05-27T07:02:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.109+0000 7f11a2a00700 10 client.15744 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.109+0000 7f11a2a00700 1 -- 172.21.15.16:0/2138691693 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560461bad570 con 0x560461d10930 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.109+0000 7f11a2a00700 10 client.15744 handle_command_reply: tid=0 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.109+0000 7f11d21bcb80 1 client.15744 shutdown 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 msgr2=0x560461c999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f11b4015b00 tx=0x7f11b4008fd0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 shutdown_connections 2024-05-27T07:02:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x560461d10930 0x560461d12d50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560461c83820 0x560461c999f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.103 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560461b04e40 0x560461c9e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 --2- 172.21.15.16:0/2138691693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560461c83410 0x560461c9ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11a2a00700 0 client.15744 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 >> 172.21.15.16:0/2138691693 conn(0x560461b64fc0 msgr2=0x560461c8d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 shutdown_connections 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.110+0000 7f11d21bcb80 1 -- 172.21.15.16:0/2138691693 wait complete. 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.111+0000 7f11d21bcb80 20 client.15744 trim_cache size 0 max 16384 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.111+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 msgr2=0x7f11a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.111+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 0x7f11a40b4f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x560461c7f690 tx=0x7f11b4009040 comp rx=0 tx=0).stop 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.111+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 msgr2=0x7f11bc1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:52.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.111+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c9e0 tx=0x7f11a800ccf0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 shutdown_connections 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f11a40b29f0 0x7f11a40b4f00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11bc109670 0x7f11bc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11bc1043f0 0x7f11bc1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 --2- 172.21.15.16:0/3193605008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11bc108d20 0x7f11bc1a0ca0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 >> 172.21.15.16:0/3193605008 conn(0x7f11bc079d00 msgr2=0x7f11bc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 shutdown_connections 2024-05-27T07:02:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:52.112+0000 7f11c2000700 1 -- 172.21.15.16:0/3193605008 wait complete. 2024-05-27T07:02:52.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:51 smithi069 ceph-mon[35276]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:52.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:02:53.509 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:53.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:53 smithi016 ceph-mon[31415]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:54.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:53 smithi164 ceph-mon[35327]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:54.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:53 smithi069 ceph-mon[35276]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:55.975 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:55 smithi016 ceph-mon[31415]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:55 smithi164 ceph-mon[35327]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:56.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:55 smithi069 ceph-mon[35276]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:56.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.379+0000 7f3e91c00700 1 -- 172.21.15.16:0/4288759675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 msgr2=0x7f3e8c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.379+0000 7f3e91c00700 1 --2- 172.21.15.16:0/4288759675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c110fd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400b0d0 tx=0x7f3e8400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 -- 172.21.15.16:0/4288759675 shutdown_connections 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 --2- 172.21.15.16:0/4288759675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c110fd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 --2- 172.21.15.16:0/4288759675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c1043f0 0x7f3e8c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 --2- 172.21.15.16:0/4288759675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c108d20 0x7f3e8c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 -- 172.21.15.16:0/4288759675 >> 172.21.15.16:0/4288759675 conn(0x7f3e8c079d00 msgr2=0x7f3e8c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 -- 172.21.15.16:0/4288759675 shutdown_connections 2024-05-27T07:02:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.380+0000 7f3e91c00700 1 -- 172.21.15.16:0/4288759675 wait complete. 2024-05-27T07:02:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.381+0000 7f3e91c00700 1 Processor -- start 2024-05-27T07:02:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.381+0000 7f3e91c00700 1 -- start start 2024-05-27T07:02:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1043f0 0x7f3e8c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 0x7f3e8c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1141a0 con 0x7f3e8c109670 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114020 con 0x7f3e8c1043f0 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114320 con 0x7f3e8c108d20 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37246/0 (socket says 172.21.15.16:37246) 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e91200700 1 -- 172.21.15.16:0/49434438 learned_addr learned my addr 172.21.15.16:0/49434438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e83400700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 0x7f3e8c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.382+0000 7f3e8be00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1043f0 0x7f3e8c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 -- 172.21.15.16:0/49434438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 msgr2=0x7f3e8c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 0x7f3e8c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 -- 172.21.15.16:0/49434438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1043f0 msgr2=0x7f3e8c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1043f0 0x7f3e8c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 -- 172.21.15.16:0/49434438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c1a1c00 con 0x7f3e8c109670 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e83400700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 0x7f3e8c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e91200700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007d10 tx=0x7f3e84007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e89600700 1 -- 172.21.15.16:0/49434438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8401f070 con 0x7f3e8c109670 2024-05-27T07:02:56.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e89600700 1 -- 172.21.15.16:0/49434438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e84003e10 con 0x7f3e8c109670 2024-05-27T07:02:56.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.383+0000 7f3e89600700 1 -- 172.21.15.16:0/49434438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8401b820 con 0x7f3e8c109670 2024-05-27T07:02:56.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.384+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c1a1e90 con 0x7f3e8c109670 2024-05-27T07:02:56.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.384+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a2320 con 0x7f3e8c109670 2024-05-27T07:02:56.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.388+0000 7f3ea1df5b80 1 Processor -- start 2024-05-27T07:02:56.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.390+0000 7f3e89600700 1 -- 172.21.15.16:0/49434438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3e84022070 con 0x7f3e8c109670 2024-05-27T07:02:56.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.390+0000 7f3ea1df5b80 1 -- start start 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.390+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baee5c9f0 0x555baeef0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.390+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baed70e40 0x555baeef0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555baee1de40 con 0x555baeefa2c0 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555baee0d9e0 con 0x555baed70e40 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555baee11a20 con 0x555baee5c9f0 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37256/0 (socket says 172.21.15.16:37256) 2024-05-27T07:02:56.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3e83400700 1 -- 172.21.15.16:0/2480270097 learned_addr learned my addr 172.21.15.16:0/2480270097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:56.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3e83400700 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baee5c9f0 0x555baeef0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.391+0000 7f3e8be00700 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baed70e40 0x555baeef0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e89600700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 0x7f3e700b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 -- 172.21.15.16:0/2480270097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baee5c9f0 msgr2=0x555baeef0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baee5c9f0 0x555baeef0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 -- 172.21.15.16:0/2480270097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baed70e40 msgr2=0x555baeef0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baed70e40 0x555baeef0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 -- 172.21.15.16:0/2480270097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baedd6420 con 0x555baeefa2c0 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e89600700 1 -- 172.21.15.16:0/49434438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f3e8400f020 con 0x7f3e8c109670 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e91200700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 0x7f3e700b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.392+0000 7f3e83400700 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005c50 tx=0x7f3e7400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e642e890cf1c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.393+0000 7f3e81600700 1 -- 172.21.15.16:0/2480270097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74004030 con 0x555baeefa2c0 2024-05-27T07:02:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.393+0000 7f3e81600700 1 -- 172.21.15.16:0/2480270097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e74003680 con 0x555baeefa2c0 2024-05-27T07:02:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.393+0000 7f3e81600700 1 -- 172.21.15.16:0/2480270097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740126b0 con 0x555baeefa2c0 2024-05-27T07:02:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3e91200700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 0x7f3e700b4f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e840e13b0 tx=0x7f3e840099e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/2480270097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 msgr2=0x555baeefc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005c50 tx=0x7f3e7400b500 comp rx=0 tx=0).stop 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/2480270097 shutdown_connections 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefc6e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baee5c9f0 0x555baeef0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/2480270097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baed70e40 0x555baeef0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/2480270097 >> 172.21.15.16:0/2480270097 conn(0x555baee0a390 msgr2=0x555baeeeb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/2480270097 shutdown_connections 2024-05-27T07:02:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.394+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/2480270097 wait complete. 2024-05-27T07:02:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.395+0000 7f3ea1df5b80 1 -- start start 2024-05-27T07:02:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.395+0000 7f3ea1df5b80 1 Processor -- start 2024-05-27T07:02:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeef72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeef7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555baed13fa0 con 0x555baeefa2c0 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555baedae760 con 0x555baee5c9f0 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555baeefce00 con 0x555baed70e40 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3ea1df5b80 10 client.0 _ll_register_callbacks cb 0x555baedb7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37264/0 (socket says 172.21.15.16:37264) 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3e83400700 1 -- 172.21.15.16:0/1041860413 learned_addr learned my addr 172.21.15.16:0/1041860413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.396+0000 7f3e83400700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeef72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeef7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 -- 172.21.15.16:0/1041860413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 msgr2=0x555baeef72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeef72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 -- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 msgr2=0x555baeef7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeef7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 -- 172.21.15.16:0/1041860413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baed5d8d0 con 0x555baeefa2c0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeef7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e83400700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3e74014690 tx=0x7f3e740145e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fc662cf93f49a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.397+0000 7f3e7a000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74005020 con 0x555baeefa2c0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f3e740051c0 con 0x555baeefa2c0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e74003680 con 0x555baeefa2c0 2024-05-27T07:02:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e740053c0 con 0x555baeefa2c0 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3ea1df5b80 10 client.15756 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555baedde060 con 0x555baeefa2c0 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.398+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e74003680 con 0x555baeefa2c0 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e740047e0 con 0x555baeefa2c0 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 20 client.15756 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef09c50 0x555baef0c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 4 client.15756 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:02:56.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.399+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555baee1fd90 con 0x555baef09c50 2024-05-27T07:02:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.400+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef09c50 0x555baef0c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.401+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef09c50 0x555baef0c070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1049ac9c68d8348 server_cookie=a0774de06f58d489 in_seq=0 out_seq=0 2024-05-27T07:02:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.401+0000 7f3e7a000700 10 client.15756 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3e7a000700 1 -- 172.21.15.16:0/1041860413 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555baee1fd90 con 0x555baef09c50 2024-05-27T07:02:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3e7a000700 10 client.15756 handle_command_reply: tid=0 2024-05-27T07:02:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 client.15756 shutdown 2024-05-27T07:02:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 msgr2=0x555baeef7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3e74014690 tx=0x7f3e740145e0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 shutdown_connections 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef09c50 0x555baef0c070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeef7d70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeef72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1041860413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeef7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 >> 172.21.15.16:0/1041860413 conn(0x555baee0a390 msgr2=0x555baeef07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3e7a000700 0 client.15756 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 shutdown_connections 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.403+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1041860413 wait complete. 2024-05-27T07:02:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.404+0000 7f3ea1df5b80 20 client.15756 trim_cache size 0 max 16384 2024-05-27T07:02:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.421+0000 7f3ea1df5b80 1 Processor -- start 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.421+0000 7f3ea1df5b80 1 -- start start 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.421+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeeeb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 0x555baef11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b2490 con 0x555baee5c9f0 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b3fd0 con 0x555baeefa2c0 2024-05-27T07:02:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b2b60 con 0x555baed70e40 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3e8be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3e8be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37278/0 (socket says 172.21.15.16:37278) 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3e8be00700 1 -- 172.21.15.16:0/1675680242 learned_addr learned my addr 172.21.15.16:0/1675680242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3e83400700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeeeb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.422+0000 7f3e83400700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 0x555baef11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 -- 172.21.15.16:0/1675680242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 msgr2=0x555baeeeb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeeeb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 -- 172.21.15.16:0/1675680242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 msgr2=0x555baef11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 0x555baef11be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 -- 172.21.15.16:0/1675680242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baeef80e0 con 0x555baee5c9f0 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e83400700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 0x555baef11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:02:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005d90 tx=0x7f3e7c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f12faf131e7866e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.423+0000 7f3e80c00700 1 -- 172.21.15.16:0/1675680242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c004030 con 0x555baee5c9f0 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.424+0000 7f3e80c00700 1 -- 172.21.15.16:0/1675680242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e7c003680 con 0x555baee5c9f0 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.424+0000 7f3e80c00700 1 -- 172.21.15.16:0/1675680242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c01d660 con 0x555baee5c9f0 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1675680242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 msgr2=0x555baef116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005d90 tx=0x7f3e7c011720 comp rx=0 tx=0).stop 2024-05-27T07:02:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1675680242 shutdown_connections 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baeefa2c0 0x555baef11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeeeb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1675680242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baee5c9f0 0x555baef116a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1675680242 >> 172.21.15.16:0/1675680242 conn(0x555baee0a390 msgr2=0x555baeeef790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1675680242 shutdown_connections 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1675680242 wait complete. 2024-05-27T07:02:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.425+0000 7f3ea1df5b80 1 -- start start 2024-05-27T07:02:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 Processor -- start 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeefd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeefd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b3900 con 0x555baeefa2c0 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b3230 con 0x555baee5c9f0 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.426+0000 7f3ea1df5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b0430 con 0x555baed70e40 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3ea1df5b80 10 client.0 _ll_register_callbacks cb 0x555baedb7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:02:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37282/0 (socket says 172.21.15.16:37282) 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 -- 172.21.15.16:0/1376645706 learned_addr learned my addr 172.21.15.16:0/1376645706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeefd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeefd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 -- 172.21.15.16:0/1376645706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 msgr2=0x555baeefd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeefd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 -- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 msgr2=0x555baeefd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeefd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e83400700 1 -- 172.21.15.16:0/1376645706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baef72610 con 0x555baeefa2c0 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.427+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeefd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3e83400700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3e74016010 tx=0x7f3e74024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccdf4f277e4bb0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3e7a000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74004e40 con 0x555baeefa2c0 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7c003680 con 0x555baeefa2c0 2024-05-27T07:02:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f3e74004fe0 con 0x555baeefa2c0 2024-05-27T07:02:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e740051e0 con 0x555baeefa2c0 2024-05-27T07:02:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3ea1df5b80 10 client.15762 fetch_fsmap learned FSMap version 31 2024-05-27T07:02:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.428+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555baef73870 con 0x555baeefa2c0 2024-05-27T07:02:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e7c003680 con 0x555baeefa2c0 2024-05-27T07:02:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e74012770 con 0x555baeefa2c0 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 20 client.15762 populate_metadata read hostname 'smithi016' 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef73e10 0x555baef76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 4 client.15762 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:02:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.429+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555baee19570 con 0x555baef73e10 2024-05-27T07:02:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.430+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef73e10 0x555baef76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.431+0000 7f3e8be00700 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef73e10 0x555baef76230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93997c1f34e4628d server_cookie=14b1f3f10b452d8b in_seq=0 out_seq=0 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.431+0000 7f3e7a000700 10 client.15762 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3e7a000700 1 -- 172.21.15.16:0/1376645706 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555baee19570 con 0x555baef73e10 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3e7a000700 10 client.15762 handle_command_reply: tid=0 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 client.15762 shutdown 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 msgr2=0x555baeefdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3e74016010 tx=0x7f3e74024040 comp rx=0 tx=0).stop 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 shutdown_connections 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555baef73e10 0x555baef76230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555baeefa2c0 0x555baeefdb00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3e7a000700 0 client.15762 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555baed70e40 0x555baeefd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 --2- 172.21.15.16:0/1376645706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555baee5c9f0 0x555baeefd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 >> 172.21.15.16:0/1376645706 conn(0x555baee0a390 msgr2=0x555baeef70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.432+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 shutdown_connections 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3ea1df5b80 1 -- 172.21.15.16:0/1376645706 wait complete. 2024-05-27T07:02:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3ea1df5b80 20 client.15762 trim_cache size 0 max 16384 2024-05-27T07:02:56.431 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:02:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 msgr2=0x7f3e700b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 0x7f3e700b4f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e840e13b0 tx=0x7f3e840099e0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 msgr2=0x7f3e8c1a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:02:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.433+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007d10 tx=0x7f3e84007df0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 shutdown_connections 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3e700b29f0 0x7f3e700b4f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e8c109670 0x7f3e8c1a1560 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1043f0 0x7f3e8c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 --2- 172.21.15.16:0/49434438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e8c108d20 0x7f3e8c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 >> 172.21.15.16:0/49434438 conn(0x7f3e8c079d00 msgr2=0x7f3e8c101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 shutdown_connections 2024-05-27T07:02:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:02:56.434+0000 7f3e91c00700 1 -- 172.21.15.16:0/49434438 wait complete. 2024-05-27T07:02:58.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:57 smithi164 ceph-mon[35327]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:58.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:57 smithi016 ceph-mon[31415]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:58.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:57 smithi069 ceph-mon[35276]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:02:58.295 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 56% (6911/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 93% (11429/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T07:02:58.602 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:02:58.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:02:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'. 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.141 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:02:59.142 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 38f9d900c83 Merge PR #55758 into wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:02:59.156 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T07:02:59.240 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T07:02:59.248 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T07:02:59.248 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T07:02:59.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:02:59 smithi016 ceph-mon[31415]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:02:59.921 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:02:59 smithi069 ceph-mon[35276]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:00.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:02:59 smithi164 ceph-mon[35327]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:01.221 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T07:03:01.517 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T07:03:01.605 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T07:03:01.610 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T07:03:01.610 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T07:03:01.690 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T07:03:01.701 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T07:03:01.701 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T07:03:01.752 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-05-27T07:03:01.754 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-05-27T07:03:01.754 DEBUG:teuthology.orchestra.run.smithi016:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-05-27T07:03:01.842 INFO:tasks.workunit.client.0.smithi016.stdout:getting iogen 2024-05-27T07:03:01.842 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'getting iogen' 2024-05-27T07:03:01.842 INFO:tasks.workunit.client.0.smithi016.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-27T07:03:01.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:01 smithi016 ceph-mon[31415]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:03:01.941 INFO:tasks.workunit.client.0.smithi016.stderr:--2024-05-27 07:03:01-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-27T07:03:01.943 INFO:tasks.workunit.client.0.smithi016.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-27T07:03:01.965 INFO:tasks.workunit.client.0.smithi016.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-27T07:03:01.987 INFO:tasks.workunit.client.0.smithi016.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-27T07:03:01.987 INFO:tasks.workunit.client.0.smithi016.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-05-27T07:03:01.995 INFO:tasks.workunit.client.0.smithi016.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-05-27T07:03:02.005 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:03:02.005 INFO:tasks.workunit.client.0.smithi016.stderr: 0K ....... 100% 230M=0s 2024-05-27T07:03:02.006 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:03:02.007 INFO:tasks.workunit.client.0.smithi016.stderr:2024-05-27 07:03:02 (230 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-05-27T07:03:02.007 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-27T07:03:02.009 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-05-27T07:03:02.036 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0 2024-05-27T07:03:02.040 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src 2024-05-27T07:03:02.046 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.8 2024-05-27T07:03:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.c 2024-05-27T07:03:02.060 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/README 2024-05-27T07:03:02.065 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile.bsd 2024-05-27T07:03:02.072 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile 2024-05-27T07:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 -- 172.21.15.16:0/2370352979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8110ab0 msgr2=0x7fe5a8112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 --2- 172.21.15.16:0/2370352979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8110ab0 0x7fe5a8112ed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009990 tx=0x7fe5a0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 -- 172.21.15.16:0/2370352979 shutdown_connections 2024-05-27T07:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 --2- 172.21.15.16:0/2370352979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8110ab0 0x7fe5a8112ed0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 --2- 172.21.15.16:0/2370352979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8102f00 0x7fe5a8103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 --2- 172.21.15.16:0/2370352979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 0x7fe5a8110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 -- 172.21.15.16:0/2370352979 >> 172.21.15.16:0/2370352979 conn(0x7fe5a8078af0 msgr2=0x7fe5a8078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 -- 172.21.15.16:0/2370352979 shutdown_connections 2024-05-27T07:03:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.085+0000 7fe5ae600700 1 -- 172.21.15.16:0/2370352979 wait complete. 2024-05-27T07:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.087+0000 7fe5ae600700 1 Processor -- start 2024-05-27T07:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.087+0000 7fe5ae600700 1 -- start start 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 0x7fe5a810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 0x7fe5a810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8116390 con 0x7fe5a8102f00 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8116210 con 0x7fe5a8110ab0 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5ae600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8116510 con 0x7fe5a8103850 2024-05-27T07:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.088+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 0x7fe5a810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58652/0 (socket says 172.21.15.16:58652) 2024-05-27T07:03:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5a7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 0x7fe5a810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5adc00700 1 -- 172.21.15.16:0/3988303653 learned_addr learned my addr 172.21.15.16:0/3988303653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:02.083 INFO:tasks.workunit.client.0.smithi016.stderr:+ cd iogen_3.1p0 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 -- 172.21.15.16:0/3988303653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 msgr2=0x7fe5a810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 0x7fe5a810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 -- 172.21.15.16:0/3988303653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 msgr2=0x7fe5a810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5ad200700 1 -- 172.21.15.16:0/3988303653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81ad290 con 0x7fe5a8102f00 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.089+0000 7fe5adc00700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.090+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 0x7fe5a810da50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007ec0 tx=0x7fe59c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.091+0000 7fe5a5600700 1 -- 172.21.15.16:0/3988303653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004db0 con 0x7fe5a8102f00 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.091+0000 7fe5a5600700 1 -- 172.21.15.16:0/3988303653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe59c005020 con 0x7fe5a8102f00 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.091+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81ad580 con 0x7fe5a8102f00 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.091+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81adb10 con 0x7fe5a8102f00 2024-05-27T07:03:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.091+0000 7fe5a5600700 1 -- 172.21.15.16:0/3988303653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c015660 con 0x7fe5a8102f00 2024-05-27T07:03:02.085 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:01 smithi164 ceph-mon[35327]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:03:02.086 INFO:tasks.workunit.client.0.smithi016.stdout:making iogen 2024-05-27T07:03:02.086 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'making iogen' 2024-05-27T07:03:02.086 INFO:tasks.workunit.client.0.smithi016.stderr:+ make 2024-05-27T07:03:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.096+0000 7fe5a5600700 1 -- 172.21.15.16:0/3988303653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe59c015800 con 0x7fe5a8102f00 2024-05-27T07:03:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.097+0000 7fe5be67ab80 1 Processor -- start 2024-05-27T07:03:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.097+0000 7fe5be67ab80 1 -- start start 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.097+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.097+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588cb3c5ec0 con 0x5588cb318ec0 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5588cb3b5a60 con 0x5588cb4a2340 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5588cb3b9aa0 con 0x5588cb404a70 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5a5600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 0x7fe5900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5a5600700 1 -- 172.21.15.16:0/3988303653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe59c0e21e0 con 0x7fe5a8102f00 2024-05-27T07:03:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5ad200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5ad200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34406/0 (socket says 172.21.15.16:34406) 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5adc00700 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5ad200700 1 -- 172.21.15.16:0/848651608 learned_addr learned my addr 172.21.15.16:0/848651608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5adc00700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 0x7fe5900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.098+0000 7fe5a7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 -- 172.21.15.16:0/848651608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 msgr2=0x5588cb4987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 -- 172.21.15.16:0/848651608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 msgr2=0x5588cb4a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 -- 172.21.15.16:0/848651608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cb37e4a0 con 0x5588cb318ec0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.099+0000 7fe5ad200700 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0df5a0 tx=0x7fe59c0738a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c887237b1e41b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.100+0000 7fe5adc00700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 0x7fe5900b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005f50 tx=0x7fe5a0005ec0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.100+0000 7fe59a000700 1 -- 172.21.15.16:0/848651608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0a8be0 con 0x5588cb318ec0 2024-05-27T07:03:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.100+0000 7fe59a000700 1 -- 172.21.15.16:0/848651608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe59c0a97d0 con 0x5588cb318ec0 2024-05-27T07:03:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.100+0000 7fe59a000700 1 -- 172.21.15.16:0/848651608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b1f30 con 0x5588cb318ec0 2024-05-27T07:03:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 -- 172.21.15.16:0/848651608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 msgr2=0x5588cb498d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0df5a0 tx=0x7fe59c0738a0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 -- 172.21.15.16:0/848651608 shutdown_connections 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/848651608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb498d10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 -- 172.21.15.16:0/848651608 >> 172.21.15.16:0/848651608 conn(0x5588cb3b2410 msgr2=0x5588cb493790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 -- 172.21.15.16:0/848651608 shutdown_connections 2024-05-27T07:03:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.101+0000 7fe5be67ab80 1 -- 172.21.15.16:0/848651608 wait complete. 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.102+0000 7fe5be67ab80 1 -- start start 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.102+0000 7fe5be67ab80 1 Processor -- start 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 0x5588cb49f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb49f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588cb2bc020 con 0x5588cb4a2340 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5588cb3567e0 con 0x5588cb318ec0 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5588cb4a4e80 con 0x5588cb404a70 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5be67ab80 10 client.0 _ll_register_callbacks cb 0x5588cb35fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34410/0 (socket says 172.21.15.16:34410) 2024-05-27T07:03:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5adc00700 1 -- 172.21.15.16:0/392908762 learned_addr learned my addr 172.21.15.16:0/392908762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5a7e00700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb49f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.103+0000 7fe5ad200700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 0x5588cb49f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 -- 172.21.15.16:0/392908762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 msgr2=0x5588cb49f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb49f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 -- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 msgr2=0x5588cb49f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 0x5588cb49f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 -- 172.21.15.16:0/392908762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588cb305950 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5ad200700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 0x5588cb49f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe5adc00700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00276d0 tx=0x7fe5a001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10ad8bd9a0e51a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.104+0000 7fe58ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0035060 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe5a0021040 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c0a97d0 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe5a003c050 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe5be67ab80 10 client.15774 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588cb3860e0 con 0x5588cb4a2340 2024-05-27T07:03:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.105+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe59c0a97d0 con 0x5588cb4a2340 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a0003b10 con 0x5588cb4a2340 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 10 client.15774 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 20 client.15774 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb4b1dd0 0x5588cb4b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 4 client.15774 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.106+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588cb3c7e10 con 0x5588cb4b1dd0 2024-05-27T07:03:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.107+0000 7fe5ad200700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb4b1dd0 0x5588cb4b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.108+0000 7fe5ad200700 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb4b1dd0 0x5588cb4b4250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47084c01c77555d server_cookie=5c03b304fdaeb36d in_seq=0 out_seq=0 2024-05-27T07:03:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.108+0000 7fe58ea00700 10 client.15774 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe58ea00700 1 -- 172.21.15.16:0/392908762 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5588cb3c7e10 con 0x5588cb4b1dd0 2024-05-27T07:03:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe58ea00700 10 client.15774 handle_command_reply: tid=0 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 client.15774 shutdown 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 msgr2=0x5588cb49fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00276d0 tx=0x7fe5a001a040 comp rx=0 tx=0).stop 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 shutdown_connections 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb4b1dd0 0x5588cb4b4250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb49fd40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb318ec0 0x5588cb49f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/392908762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb49f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 >> 172.21.15.16:0/392908762 conn(0x5588cb3b2410 msgr2=0x5588cb4932e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe58ea00700 0 client.15774 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.112+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 shutdown_connections 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.113+0000 7fe5be67ab80 1 -- 172.21.15.16:0/392908762 wait complete. 2024-05-27T07:03:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.113+0000 7fe5be67ab80 20 client.15774 trim_cache size 0 max 16384 2024-05-27T07:03:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:01 smithi069 ceph-mon[35276]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T07:03:02.122 INFO:tasks.workunit.client.0.smithi016.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-05-27T07:03:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.139+0000 7fe5be67ab80 1 Processor -- start 2024-05-27T07:03:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.139+0000 7fe5be67ab80 1 -- start start 2024-05-27T07:03:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb318ec0 0x5588cb49e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb404a70 0x5588cb49a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe58800ecd0 con 0x5588cb4a2340 2024-05-27T07:03:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe588010810 con 0x5588cb404a70 2024-05-27T07:03:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe58800f3a0 con 0x5588cb318ec0 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5adc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34422/0 (socket says 172.21.15.16:34422) 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5adc00700 1 -- 172.21.15.16:0/2417609426 learned_addr learned my addr 172.21.15.16:0/2417609426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5ad200700 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb318ec0 0x5588cb49e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.140+0000 7fe5a7e00700 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb404a70 0x5588cb49a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 -- 172.21.15.16:0/2417609426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb318ec0 msgr2=0x5588cb49e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb318ec0 0x5588cb49e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 -- 172.21.15.16:0/2417609426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb404a70 msgr2=0x5588cb49a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb404a70 0x5588cb49a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 -- 172.21.15.16:0/2417609426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cb494080 con 0x5588cb4a2340 2024-05-27T07:03:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.141+0000 7fe5adc00700 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000b410 tx=0x7fe5a0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eda1d59a811fbba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.142+0000 7fe599600700 1 -- 172.21.15.16:0/2417609426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0035040 con 0x5588cb4a2340 2024-05-27T07:03:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.142+0000 7fe599600700 1 -- 172.21.15.16:0/2417609426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a00305b0 con 0x5588cb4a2340 2024-05-27T07:03:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.142+0000 7fe599600700 1 -- 172.21.15.16:0/2417609426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003990 con 0x5588cb4a2340 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 -- 172.21.15.16:0/2417609426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 msgr2=0x5588cb4adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000b410 tx=0x7fe5a0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 -- 172.21.15.16:0/2417609426 shutdown_connections 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb4a2340 0x5588cb4adf50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb318ec0 0x5588cb49e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/2417609426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb404a70 0x5588cb49a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 -- 172.21.15.16:0/2417609426 >> 172.21.15.16:0/2417609426 conn(0x5588cb3b2410 msgr2=0x5588cb49a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 -- 172.21.15.16:0/2417609426 shutdown_connections 2024-05-27T07:03:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.143+0000 7fe5be67ab80 1 -- 172.21.15.16:0/2417609426 wait complete. 2024-05-27T07:03:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.144+0000 7fe5be67ab80 1 -- start start 2024-05-27T07:03:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.144+0000 7fe5be67ab80 1 Processor -- start 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.144+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe588010140 con 0x5588cb318ec0 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe58800fa70 con 0x5588cb4a2340 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe58800cc70 con 0x5588cb404a70 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5be67ab80 10 client.0 _ll_register_callbacks cb 0x5588cb35fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34426/0 (socket says 172.21.15.16:34426) 2024-05-27T07:03:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 -- 172.21.15.16:0/3786590701 learned_addr learned my addr 172.21.15.16:0/3786590701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5a7e00700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5adc00700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 -- 172.21.15.16:0/3786590701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 msgr2=0x5588cb4a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 -- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 msgr2=0x5588cb4a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.145+0000 7fe5ad200700 1 -- 172.21.15.16:0/3786590701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588cb4a6b40 con 0x5588cb318ec0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.146+0000 7fe5ad200700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0a8dd0 tx=0x7fe59c0a8fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff92becc7811a4e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.146+0000 7fe58ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.146+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b1f30 con 0x5588cb318ec0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.146+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fe59c0df7c0 con 0x5588cb318ec0 2024-05-27T07:03:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.146+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a00305b0 con 0x5588cb318ec0 2024-05-27T07:03:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe59c0df9c0 con 0x5588cb318ec0 2024-05-27T07:03:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 10 client.15780 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588cb51c9b0 con 0x5588cb318ec0 2024-05-27T07:03:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a00305b0 con 0x5588cb318ec0 2024-05-27T07:03:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe59c0a93c0 con 0x5588cb318ec0 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 10 client.15780 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 20 client.15780 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb51cc40 0x5588cb51f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 4 client.15780 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.147+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5588cb3c15f0 con 0x5588cb51cc40 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.148+0000 7fe5a7e00700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb51cc40 0x5588cb51f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.149+0000 7fe5a7e00700 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb51cc40 0x5588cb51f060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a16328786890c server_cookie=9af344bac67ff98a in_seq=0 out_seq=0 2024-05-27T07:03:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.149+0000 7fe58ea00700 10 client.15780 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.152+0000 7fe58ea00700 1 -- 172.21.15.16:0/3786590701 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5588cb3c15f0 con 0x5588cb51cc40 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.152+0000 7fe58ea00700 10 client.15780 handle_command_reply: tid=0 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.152+0000 7fe5be67ab80 1 client.15780 shutdown 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 msgr2=0x5588cb4aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0a8dd0 tx=0x7fe59c0a8fd0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 shutdown_connections 2024-05-27T07:03:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5588cb51cc40 0x5588cb51f060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588cb4a2340 0x5588cb4a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588cb318ec0 0x5588cb4aa9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 --2- 172.21.15.16:0/3786590701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588cb404a70 0x5588cb4a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 >> 172.21.15.16:0/3786590701 conn(0x5588cb3b2410 msgr2=0x5588cb4971a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe58ea00700 0 client.15780 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 shutdown_connections 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.153+0000 7fe5be67ab80 1 -- 172.21.15.16:0/3786590701 wait complete. 2024-05-27T07:03:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.154+0000 7fe5be67ab80 20 client.15780 trim_cache size 0 max 16384 2024-05-27T07:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "90ed5088-c65e-4896-b6cf-c709833025fc", 2024-05-27T07:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.154+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 msgr2=0x7fe5900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.154+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 0x7fe5900b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005f50 tx=0x7fe5a0005ec0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 msgr2=0x7fe5a810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 0x7fe5a810da50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007ec0 tx=0x7fe59c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 shutdown_connections 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fe5900b2a70 0x7fe5900b4f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5a8110ab0 0x7fe5a810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5a8102f00 0x7fe5a810da50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 --2- 172.21.15.16:0/3988303653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8103850 0x7fe5a810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 >> 172.21.15.16:0/3988303653 conn(0x7fe5a8078af0 msgr2=0x7fe5a8111f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 shutdown_connections 2024-05-27T07:03:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:02.155+0000 7fe5ae600700 1 -- 172.21.15.16:0/3988303653 wait complete. 2024-05-27T07:03:02.902 INFO:tasks.workunit.client.0.smithi016.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-05-27T07:03:03.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:03:03.292 INFO:tasks.workunit.client.0.smithi016.stdout:running iogen 2024-05-27T07:03:03.292 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'running iogen' 2024-05-27T07:03:03.292 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -n 5 -s 2g 2024-05-27T07:03:03.297 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 96256 2024-05-27T07:03:03.297 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 96258 2024-05-27T07:03:03.297 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 96259 2024-05-27T07:03:03.298 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 96260 2024-05-27T07:03:03.299 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 96262 2024-05-27T07:03:03.299 INFO:tasks.workunit.client.0.smithi016.stdout:sleep for 10 min 2024-05-27T07:03:03.299 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'sleep for 10 min' 2024-05-27T07:03:03.300 INFO:tasks.workunit.client.0.smithi016.stderr:+ sleep 600 2024-05-27T07:03:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:04.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:03 smithi016 ceph-mon[31415]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:04.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:03 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:03:04.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:03 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:03 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:03:04.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:03 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:03 smithi164 ceph-mon[35327]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:04.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:03 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:03:04.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:03 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:03 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:03:04.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:03 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:03 smithi069 ceph-mon[35276]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:04.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:03 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:03:04.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:03 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:03 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:03:04.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:03 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:04.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:04 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:03:05.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:04 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:03:05.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:04 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:03:05.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:05 smithi016 ceph-mon[31415]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:06.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:05 smithi164 ceph-mon[35327]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:06.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:05 smithi069 ceph-mon[35276]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.558+0000 7f23c2c00700 1 -- 172.21.15.16:0/2165193194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 msgr2=0x7f23bc113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.558+0000 7f23c2c00700 1 --2- 172.21.15.16:0/2165193194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc113340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f23b400b0a0 tx=0x7f23b400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.559+0000 7f23c2c00700 1 -- 172.21.15.16:0/2165193194 shutdown_connections 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.559+0000 7f23c2c00700 1 --2- 172.21.15.16:0/2165193194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc113340 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.559+0000 7f23c2c00700 1 --2- 172.21.15.16:0/2165193194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10c5a0 0x7f23bc10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.559+0000 7f23c2c00700 1 --2- 172.21.15.16:0/2165193194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23bc106560 0x7f23bc106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.559+0000 7f23c2c00700 1 -- 172.21.15.16:0/2165193194 >> 172.21.15.16:0/2165193194 conn(0x7f23bc102040 msgr2=0x7f23bc104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.560+0000 7f23c2c00700 1 -- 172.21.15.16:0/2165193194 shutdown_connections 2024-05-27T07:03:06.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.561+0000 7f23c2c00700 1 -- 172.21.15.16:0/2165193194 wait complete. 2024-05-27T07:03:06.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.562+0000 7f23c2c00700 1 Processor -- start 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.563+0000 7f23c2c00700 1 -- start start 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.563+0000 7f23c2c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 0x7f23bc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.563+0000 7f23c2c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.563+0000 7f23c2c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23bc10c5a0 0x7f23bc1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23c2c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1165d0 con 0x7f23bc106eb0 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23c2c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc116450 con 0x7f23bc106560 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23c2c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc116750 con 0x7f23bc10c5a0 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 0x7f23bc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 0x7f23bc19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58702/0 (socket says 172.21.15.16:58702) 2024-05-27T07:03:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.564+0000 7f23bbe00700 1 -- 172.21.15.16:0/834058120 learned_addr learned my addr 172.21.15.16:0/834058120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.565+0000 7f23bb400700 1 -- 172.21.15.16:0/834058120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23bc10c5a0 msgr2=0x7f23bc1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.565+0000 7f23bb400700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23bc10c5a0 0x7f23bc1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.565+0000 7f23bb400700 1 -- 172.21.15.16:0/834058120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 msgr2=0x7f23bc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.565+0000 7f23bb400700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 0x7f23bc19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.565+0000 7f23bb400700 1 -- 172.21.15.16:0/834058120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc1a1e90 con 0x7f23bc106eb0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.567+0000 7f23bb400700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc19d560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f23bc06c610 tx=0x7f23a800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.567+0000 7f23b8c00700 1 -- 172.21.15.16:0/834058120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8017070 con 0x7f23bc106eb0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.567+0000 7f23b8c00700 1 -- 172.21.15.16:0/834058120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a8009d90 con 0x7f23bc106eb0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.567+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc1a2180 con 0x7f23bc106eb0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.567+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc06d1d0 con 0x7f23bc106eb0 2024-05-27T07:03:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.568+0000 7f23b8c00700 1 -- 172.21.15.16:0/834058120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8010df0 con 0x7f23bc106eb0 2024-05-27T07:03:06.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.573+0000 7f23b8c00700 1 -- 172.21.15.16:0/834058120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f23a801a070 con 0x7f23bc106eb0 2024-05-27T07:03:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.575+0000 7f23b8c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.575+0000 7f23d16fdb80 1 Processor -- start 2024-05-27T07:03:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23b8c00700 1 -- 172.21.15.16:0/834058120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f23a80e2460 con 0x7f23bc106eb0 2024-05-27T07:03:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 -- start start 2024-05-27T07:03:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 0x558f3c036330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 0x558f3c043110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3bf64e40 con 0x558f3beb7e40 2024-05-27T07:03:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f3bf549e0 con 0x558f3bfa39f0 2024-05-27T07:03:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558f3bf58a20 con 0x558f3c036e70 2024-05-27T07:03:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23bbe00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.576+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34452/0 (socket says 172.21.15.16:34452) 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 -- 172.21.15.16:0/2041587464 learned_addr learned my addr 172.21.15.16:0/2041587464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bbe00700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 0x558f3c036330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23c0c00700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 0x558f3c043110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 -- 172.21.15.16:0/2041587464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 msgr2=0x558f3c043110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 0x558f3c043110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 -- 172.21.15.16:0/2041587464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 msgr2=0x558f3c036330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 0x558f3c036330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bb400700 1 -- 172.21.15.16:0/2041587464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3bf1d420 con 0x558f3beb7e40 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23c0c00700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 0x558f3c043110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.577+0000 7f23bbe00700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 0x558f3c036330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.578+0000 7f23bb400700 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f23a80a66c0 tx=0x7f23a80a8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7249cadccc25785c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.578+0000 7f23bbe00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 0x7f23a00b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x558f3c038ff0 tx=0x7f23ac00b820 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.579+0000 7f23b1600700 1 -- 172.21.15.16:0/2041587464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80ad800 con 0x558f3beb7e40 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.579+0000 7f23b1600700 1 -- 172.21.15.16:0/2041587464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a80b22b0 con 0x558f3beb7e40 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.579+0000 7f23b1600700 1 -- 172.21.15.16:0/2041587464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80a98d0 con 0x558f3beb7e40 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2041587464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 msgr2=0x558f3c036870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f23a80a66c0 tx=0x7f23a80a8f60 comp rx=0 tx=0).stop 2024-05-27T07:03:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2041587464 shutdown_connections 2024-05-27T07:03:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3c036e70 0x558f3c043110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3bfa39f0 0x558f3c036330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2041587464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3beb7e40 0x558f3c036870 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2041587464 >> 172.21.15.16:0/2041587464 conn(0x558f3bf51390 msgr2=0x558f3c0324c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2041587464 shutdown_connections 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.580+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2041587464 wait complete. 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.581+0000 7f23d16fdb80 1 -- start start 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.581+0000 7f23d16fdb80 1 Processor -- start 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.582+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c040680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.582+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.582+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c036e70 0x558f3c041100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.583+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3be5afa0 con 0x558f3bfa39f0 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.583+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.583+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34464/0 (socket says 172.21.15.16:34464) 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.583+0000 7f23bb400700 1 -- 172.21.15.16:0/2605761508 learned_addr learned my addr 172.21.15.16:0/2605761508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.584+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f3bef5760 con 0x558f3c036e70 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.584+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558f3c043b80 con 0x558f3beb7e40 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.584+0000 7f23c0c00700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c036e70 0x558f3c041100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.584+0000 7f23d16fdb80 10 client.0 _ll_register_callbacks cb 0x558f3befec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bbe00700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c040680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bb400700 1 -- 172.21.15.16:0/2605761508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 msgr2=0x558f3c040680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bb400700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c040680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bb400700 1 -- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c036e70 msgr2=0x558f3c041100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bb400700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c036e70 0x558f3c041100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.585+0000 7f23bb400700 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3bea48d0 con 0x558f3bfa39f0 2024-05-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.586+0000 7f23bb400700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23a80df970 tx=0x7f23a80e23a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dcba7ded3dc532 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.586+0000 7f23a6000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.587+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80a96b0 con 0x558f3bfa39f0 2024-05-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.587+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f23a80a9850 con 0x558f3bfa39f0 2024-05-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.587+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a80b22b0 con 0x558f3bfa39f0 2024-05-27T07:03:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.588+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f23a80b1e20 con 0x558f3bfa39f0 2024-05-27T07:03:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.588+0000 7f23d16fdb80 10 client.15792 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.588+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f3bf25060 con 0x558f3bfa39f0 2024-05-27T07:03:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.588+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a80b22b0 con 0x558f3bfa39f0 2024-05-27T07:03:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.588+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f23a80a86b0 con 0x558f3bfa39f0 2024-05-27T07:03:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 20 client.15792 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c059160 0x558f3c05b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 4 client.15792 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.589+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3bf66d90 con 0x558f3c059160 2024-05-27T07:03:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.590+0000 7f23bbe00700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c059160 0x558f3c05b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.591+0000 7f23a6000700 10 client.15792 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.591+0000 7f23bbe00700 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c059160 0x558f3c05b580 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc4005806d8e776 server_cookie=d3a7900d6d29f5ff in_seq=0 out_seq=0 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.593+0000 7f23a6000700 1 -- 172.21.15.16:0/2605761508 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558f3bf66d90 con 0x558f3c059160 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.593+0000 7f23a6000700 10 client.15792 handle_command_reply: tid=0 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.593+0000 7f23d16fdb80 1 client.15792 shutdown 2024-05-27T07:03:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 msgr2=0x558f3c040bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23a80df970 tx=0x7f23a80e23a0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 shutdown_connections 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c059160 0x558f3c05b580 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c036e70 0x558f3c041100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.594+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c040680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/2605761508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3bfa39f0 0x558f3c040bc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23a6000700 0 client.15792 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 >> 172.21.15.16:0/2605761508 conn(0x558f3bf51390 msgr2=0x558f3c036310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 shutdown_connections 2024-05-27T07:03:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23d16fdb80 1 -- 172.21.15.16:0/2605761508 wait complete. 2024-05-27T07:03:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.595+0000 7f23d16fdb80 20 client.15792 trim_cache size 0 max 16384 2024-05-27T07:03:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.626+0000 7f23d16fdb80 1 Processor -- start 2024-05-27T07:03:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.626+0000 7f23d16fdb80 1 -- start start 2024-05-27T07:03:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.627+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.627+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 0x558f3c055710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c038900 0x558f3c032530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00ee50 con 0x558f3c036e70 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f239c010990 con 0x558f3c038900 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00f520 con 0x558f3beb7e40 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 0x558f3c055710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59410/0 (socket says 172.21.15.16:59410) 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.628+0000 7f23bbe00700 1 -- 172.21.15.16:0/1130689654 learned_addr learned my addr 172.21.15.16:0/1130689654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23c0c00700 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c038900 0x558f3c032530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bb400700 1 -- 172.21.15.16:0/1130689654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 msgr2=0x558f3c03f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bb400700 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bb400700 1 -- 172.21.15.16:0/1130689654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c038900 msgr2=0x558f3c032530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bb400700 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c038900 0x558f3c032530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bb400700 1 -- 172.21.15.16:0/1130689654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3c0339c0 con 0x558f3c036e70 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.629+0000 7f23bbe00700 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.630+0000 7f23bb400700 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 0x558f3c055710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x558f3c041c50 tx=0x7f23a80a8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5defccba5d16045d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.631+0000 7f23b0c00700 1 -- 172.21.15.16:0/1130689654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80b22b0 con 0x558f3c036e70 2024-05-27T07:03:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.631+0000 7f23b0c00700 1 -- 172.21.15.16:0/1130689654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a80b1fc0 con 0x558f3c036e70 2024-05-27T07:03:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.631+0000 7f23b0c00700 1 -- 172.21.15.16:0/1130689654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80a86b0 con 0x558f3c036e70 2024-05-27T07:03:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 -- 172.21.15.16:0/1130689654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 msgr2=0x558f3c055710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 0x558f3c055710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x558f3c041c50 tx=0x7f23a80a8f60 comp rx=0 tx=0).stop 2024-05-27T07:03:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 -- 172.21.15.16:0/1130689654 shutdown_connections 2024-05-27T07:03:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c038900 0x558f3c032530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c03f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/1130689654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c036e70 0x558f3c055710 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x558f3c041c50 tx=0x7f23a80a8f60 comp rx=0 tx=0).stop 2024-05-27T07:03:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.632+0000 7f23d16fdb80 1 -- 172.21.15.16:0/1130689654 >> 172.21.15.16:0/1130689654 conn(0x558f3bf51390 msgr2=0x558f3c041d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.633+0000 7f23d16fdb80 1 -- 172.21.15.16:0/1130689654 shutdown_connections 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.633+0000 7f23d16fdb80 1 -- 172.21.15.16:0/1130689654 wait complete. 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.634+0000 7f23d16fdb80 1 -- start start 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.634+0000 7f23d16fdb80 1 Processor -- start 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c05a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c05afc0 0x558f3c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f239c0102c0 con 0x558f3c038900 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00fbf0 con 0x558f3c05afc0 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.635+0000 7f23d16fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00cdf0 con 0x558f3beb7e40 2024-05-27T07:03:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.636+0000 7f23d16fdb80 10 client.0 _ll_register_callbacks cb 0x558f3befec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.636+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.636+0000 7f23bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2024-05-27T07:03:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.636+0000 7f23bb400700 1 -- 172.21.15.16:0/51589835 learned_addr learned my addr 172.21.15.16:0/51589835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 -- 172.21.15.16:0/51589835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 msgr2=0x558f3c05a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c05a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 -- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c05afc0 msgr2=0x558f3c0c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c05afc0 0x558f3c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 -- 172.21.15.16:0/51589835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3c0c2fd0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23bb400700 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f23a80a8a90 tx=0x7f23a80a8c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062d56fefca653f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.637+0000 7f23a6000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.638+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80b22b0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.638+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f23a80a96b0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.638+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a80b1fc0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.639+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f23a80a98f0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.639+0000 7f23d16fdb80 10 client.15798 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.639+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f3c0c2fd0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.639+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a80b1fc0 con 0x558f3c038900 2024-05-27T07:03:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.639+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f23a8009a10 con 0x558f3c038900 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 10 client.15798 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 20 client.15798 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c0c4430 0x558f3c0c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 4 client.15798 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f3bf60570 con 0x558f3c0c4430 2024-05-27T07:03:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.640+0000 7f23bbe00700 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c0c4430 0x558f3c0c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.642+0000 7f23bbe00700 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c0c4430 0x558f3c0c6850 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b083b5406c4ef8 server_cookie=cafc7116dadad65a in_seq=0 out_seq=0 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.642+0000 7f23a6000700 10 client.15798 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.642+0000 7f23a6000700 1 -- 172.21.15.16:0/51589835 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558f3bf60570 con 0x558f3c0c4430 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.642+0000 7f23a6000700 10 client.15798 handle_command_reply: tid=0 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.642+0000 7f23d16fdb80 1 client.15798 shutdown 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 msgr2=0x558f3c05aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f23a80a8a90 tx=0x7f23a80a8c70 comp rx=0 tx=0).stop 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 shutdown_connections 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x558f3c0c4430 0x558f3c0c6850 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f3c05afc0 0x558f3c0c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558f3beb7e40 0x558f3c05a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 --2- 172.21.15.16:0/51589835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3c038900 0x558f3c05aa80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 >> 172.21.15.16:0/51589835 conn(0x558f3bf51390 msgr2=0x558f3c040e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23a6000700 0 client.15798 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 shutdown_connections 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 1 -- 172.21.15.16:0/51589835 wait complete. 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.643+0000 7f23d16fdb80 20 client.15798 trim_cache size 0 max 16384 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:03:06.644 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:03:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 msgr2=0x7f23a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 0x7f23a00b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x558f3c038ff0 tx=0x7f23ac00b820 comp rx=0 tx=0).stop 2024-05-27T07:03:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 msgr2=0x7f23bc19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc19d560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f23bc06c610 tx=0x7f23a800dca0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 shutdown_connections 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23bc10c5a0 0x7f23bc1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc106560 0x7f23bc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 --2- 172.21.15.16:0/834058120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23bc106eb0 0x7f23bc19d560 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.645+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 >> 172.21.15.16:0/834058120 conn(0x7f23bc102040 msgr2=0x7f23bc103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.646+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 shutdown_connections 2024-05-27T07:03:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:06.646+0000 7f23c2c00700 1 -- 172.21.15.16:0/834058120 wait complete. 2024-05-27T07:03:07.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:03:07.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:03:07.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:07 smithi016 ceph-mon[31415]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 36 op/s 2024-05-27T07:03:08.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:07 smithi164 ceph-mon[35327]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 36 op/s 2024-05-27T07:03:08.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:07 smithi069 ceph-mon[35276]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 36 op/s 2024-05-27T07:03:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:09 smithi164 ceph-mon[35327]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 32 op/s 2024-05-27T07:03:10.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:09 smithi016 ceph-mon[31415]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 32 op/s 2024-05-27T07:03:10.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:09 smithi069 ceph-mon[35276]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 32 op/s 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 -- 172.21.15.16:0/3314536625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5624115360 msgr2=0x7f5624117780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 --2- 172.21.15.16:0/3314536625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5624115360 0x7f5624117780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f561c00b0a0 tx=0x7f561c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 -- 172.21.15.16:0/3314536625 shutdown_connections 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 --2- 172.21.15.16:0/3314536625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5624115360 0x7f5624117780 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 --2- 172.21.15.16:0/3314536625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f5624075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 --2- 172.21.15.16:0/3314536625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56240764a0 0x7f5624114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 -- 172.21.15.16:0/3314536625 >> 172.21.15.16:0/3314536625 conn(0x7f5624100030 msgr2=0x7f5624102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.882+0000 7f562bc00700 1 -- 172.21.15.16:0/3314536625 shutdown_connections 2024-05-27T07:03:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.883+0000 7f562bc00700 1 -- 172.21.15.16:0/3314536625 wait complete. 2024-05-27T07:03:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.884+0000 7f562bc00700 1 Processor -- start 2024-05-27T07:03:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.884+0000 7f562bc00700 1 -- start start 2024-05-27T07:03:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.884+0000 7f562bc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f56241a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.884+0000 7f562bc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 0x7f56241a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f562bc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5624115360 0x7f56241a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f562bc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f562411a960 con 0x7f56240764a0 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f562bc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f562411a7e0 con 0x7f5624075b50 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f562bc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f562411aae0 con 0x7f5624115360 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f5629200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f56241a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f5623e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 0x7f56241a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f5623e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 0x7f56241a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34494/0 (socket says 172.21.15.16:34494) 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f5623e00700 1 -- 172.21.15.16:0/892323370 learned_addr learned my addr 172.21.15.16:0/892323370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.885+0000 7f5629c00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5624115360 0x7f56241a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.886+0000 7f5629200700 1 -- 172.21.15.16:0/892323370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5624115360 msgr2=0x7f56241a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.886+0000 7f5629200700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5624115360 0x7f56241a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.886+0000 7f5629200700 1 -- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 msgr2=0x7f56241a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.886+0000 7f5629200700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 0x7f56241a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.886+0000 7f5629200700 1 -- 172.21.15.16:0/892323370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56241a6200 con 0x7f5624075b50 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f5629200700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f56241a1330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f561800d950 tx=0x7f561800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618016070 con 0x7f5624075b50 2024-05-27T07:03:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56241a64f0 con 0x7f5624075b50 2024-05-27T07:03:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241ade10 con 0x7f5624075b50 2024-05-27T07:03:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561800fd20 con 0x7f5624075b50 2024-05-27T07:03:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.887+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561800e3b0 con 0x7f5624075b50 2024-05-27T07:03:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.890+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604005330 con 0x7f5624075b50 2024-05-27T07:03:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.894+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5618019070 con 0x7f5624075b50 2024-05-27T07:03:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.896+0000 7f5621600700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 0x7f560c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.897+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f56180e27b0 con 0x7f5624075b50 2024-05-27T07:03:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.898+0000 7f5623e00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 0x7f560c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.900+0000 7f5623e00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 0x7f560c0b5000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56241034e0 tx=0x7f5610007480 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:10.915+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56180a9da0 con 0x7f5624075b50 2024-05-27T07:03:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.388+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5604005620 con 0x7f5624075b50 2024-05-27T07:03:11.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:03:11.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:03:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.390+0000 7f5621600700 1 -- 172.21.15.16:0/892323370 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f56180a96a0 con 0x7f5624075b50 2024-05-27T07:03:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 msgr2=0x7f560c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 0x7f560c0b5000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56241034e0 tx=0x7f5610007480 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 msgr2=0x7f56241a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f56241a1330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f561800d950 tx=0x7f561800dc60 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 shutdown_connections 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f560c0b2af0 0x7f560c0b5000 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5624115360 0x7f56241a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5624075b50 0x7f56241a1330 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f561800d950 tx=0x7f561800dc60 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 --2- 172.21.15.16:0/892323370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56240764a0 0x7f56241a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.394+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 >> 172.21.15.16:0/892323370 conn(0x7f5624100030 msgr2=0x7f56241006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:11.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.395+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 shutdown_connections 2024-05-27T07:03:11.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:11.396+0000 7f562bc00700 1 -- 172.21.15.16:0/892323370 wait complete. 2024-05-27T07:03:11.390 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:03:11.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:11 smithi016 ceph-mon[31415]: pgmap v71: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 51 MiB/s wr, 60 op/s 2024-05-27T07:03:11.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:11 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/892323370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:12.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:11 smithi164 ceph-mon[35327]: pgmap v71: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 51 MiB/s wr, 60 op/s 2024-05-27T07:03:12.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:11 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/892323370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:12.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:11 smithi069 ceph-mon[35276]: pgmap v71: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 51 MiB/s wr, 60 op/s 2024-05-27T07:03:12.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:11 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/892323370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:12.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:03:12.942 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:13 smithi016 ceph-mon[31415]: pgmap v72: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:14.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:13 smithi164 ceph-mon[35327]: pgmap v72: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:14.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:13 smithi069 ceph-mon[35276]: pgmap v72: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:16.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:15 smithi164 ceph-mon[35327]: pgmap v73: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:16.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:15 smithi016 ceph-mon[31415]: pgmap v73: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:16.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:15 smithi069 ceph-mon[35276]: pgmap v73: 129 pgs: 129 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 56 op/s 2024-05-27T07:03:17.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:16 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:03:17.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:16 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:03:17.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:16 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:03:17.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:17 smithi016 ceph-mon[31415]: pgmap v74: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 94 MiB/s wr, 82 op/s 2024-05-27T07:03:18.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:17 smithi164 ceph-mon[35327]: pgmap v74: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 94 MiB/s wr, 82 op/s 2024-05-27T07:03:18.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:17 smithi069 ceph-mon[35276]: pgmap v74: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 94 MiB/s wr, 82 op/s 2024-05-27T07:03:18.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 -- 172.21.15.16:0/709116950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c1192b0 msgr2=0x7f7b9c11b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 --2- 172.21.15.16:0/709116950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c11b7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400b0a0 tx=0x7f7b9400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 -- 172.21.15.16:0/709116950 shutdown_connections 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 --2- 172.21.15.16:0/709116950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c11b7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 --2- 172.21.15.16:0/709116950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 --2- 172.21.15.16:0/709116950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 0x7f7b9c118c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.463+0000 7f7ba3600700 1 -- 172.21.15.16:0/709116950 >> 172.21.15.16:0/709116950 conn(0x7f7b9c06f4c0 msgr2=0x7f7b9c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.465+0000 7f7ba3600700 1 -- 172.21.15.16:0/709116950 shutdown_connections 2024-05-27T07:03:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.465+0000 7f7ba3600700 1 -- 172.21.15.16:0/709116950 wait complete. 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.470+0000 7f7ba3600700 1 Processor -- start 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.470+0000 7f7ba3600700 1 -- start start 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.471+0000 7f7ba3600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c116000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.471+0000 7f7ba3600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 0x7f7b9c116540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.472+0000 7f7ba3600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c116a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.472+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c116000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.472+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c116000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59870/0 (socket says 172.21.15.16:59870) 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.472+0000 7f7ba3600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c11ed50 con 0x7f7b9c10b500 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.472+0000 7f7ba0c00700 1 -- 172.21.15.16:0/1696097927 learned_addr learned my addr 172.21.15.16:0/1696097927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.473+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c11ebd0 con 0x7f7b9c1192b0 2024-05-27T07:03:18.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.473+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c11eed0 con 0x7f7b9c10abb0 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.473+0000 7f7b9be00700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 0x7f7b9c116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.473+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 -- 172.21.15.16:0/1696097927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 msgr2=0x7f7b9c116000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c116000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 -- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 msgr2=0x7f7b9c116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 0x7f7b9c116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 -- 172.21.15.16:0/1696097927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1b5bf0 con 0x7f7b9c1192b0 2024-05-27T07:03:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.475+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c116a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400b0a0 tx=0x7f7b94004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.479+0000 7f7b99600700 1 -- 172.21.15.16:0/1696097927 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94021070 con 0x7f7b9c1192b0 2024-05-27T07:03:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.479+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1b5e80 con 0x7f7b9c1192b0 2024-05-27T07:03:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.479+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1b6330 con 0x7f7b9c1192b0 2024-05-27T07:03:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.479+0000 7f7b99600700 1 -- 172.21.15.16:0/1696097927 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b94007c40 con 0x7f7b9c1192b0 2024-05-27T07:03:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.479+0000 7f7b99600700 1 -- 172.21.15.16:0/1696097927 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940052a0 con 0x7f7b9c1192b0 2024-05-27T07:03:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.488+0000 7f7b99600700 1 -- 172.21.15.16:0/1696097927 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7b94024070 con 0x7f7b9c1192b0 2024-05-27T07:03:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.492+0000 7f7bb21c1b80 1 Processor -- start 2024-05-27T07:03:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 -- start start 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45159f0 0x563bf45a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bf44d6e40 con 0x563bf45a8e70 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bf44c69e0 con 0x563bf4429e40 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bf44caa20 con 0x563bf45159f0 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.493+0000 7f7b99600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 0x7f7b840b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.494+0000 7f7b99600700 1 -- 172.21.15.16:0/1696097927 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7b9400f020 con 0x7f7b9c1192b0 2024-05-27T07:03:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.494+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 0x7f7b840b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.494+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.494+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:33938/0 (socket says 172.21.15.16:33938) 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.494+0000 7f7ba0c00700 1 -- 172.21.15.16:0/664536160 learned_addr learned my addr 172.21.15.16:0/664536160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.495+0000 7f7ba1600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 0x7f7b840b5000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b940a6e60 tx=0x7f7b94009040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.496+0000 7f7ba0c00700 1 -- 172.21.15.16:0/664536160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45159f0 msgr2=0x563bf45a8330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.496+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45159f0 0x563bf45a8330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.496+0000 7f7ba0c00700 1 -- 172.21.15.16:0/664536160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 msgr2=0x563bf45b5110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:03:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.498+0000 7f7b9be00700 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.498+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.498+0000 7f7ba0c00700 1 -- 172.21.15.16:0/664536160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf448f420 con 0x563bf4429e40 2024-05-27T07:03:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.499+0000 7f7b9be00700 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.499+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009dc0 tx=0x7f7b8c0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d50883f0f2ccf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.500+0000 7f7b8a000700 1 -- 172.21.15.16:0/664536160 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c012450 con 0x563bf4429e40 2024-05-27T07:03:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.500+0000 7f7b8a000700 1 -- 172.21.15.16:0/664536160 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b8c012a80 con 0x563bf4429e40 2024-05-27T07:03:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.500+0000 7f7b8a000700 1 -- 172.21.15.16:0/664536160 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00f920 con 0x563bf4429e40 2024-05-27T07:03:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.503+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/664536160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 msgr2=0x563bf45a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.503+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009dc0 tx=0x7f7b8c0116c0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.507+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/664536160 shutdown_connections 2024-05-27T07:03:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.507+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.507+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45159f0 0x563bf45a8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.507+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/664536160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.507+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/664536160 >> 172.21.15.16:0/664536160 conn(0x563bf44c3390 msgr2=0x563bf45a44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.508+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/664536160 shutdown_connections 2024-05-27T07:03:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.509+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/664536160 wait complete. 2024-05-27T07:03:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.511+0000 7f7bb21c1b80 1 -- start start 2024-05-27T07:03:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.512+0000 7f7bb21c1b80 1 Processor -- start 2024-05-27T07:03:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf4429e40 0x563bf45b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bf43ccfa0 con 0x563bf4429e40 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bf4467760 con 0x563bf45159f0 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bf45b5b80 con 0x563bf45a8e70 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.513+0000 7f7bb21c1b80 10 client.0 _ll_register_callbacks cb 0x563bf4470c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.514+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.514+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:33950/0 (socket says 172.21.15.16:33950) 2024-05-27T07:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.514+0000 7f7ba0c00700 1 -- 172.21.15.16:0/2190936239 learned_addr learned my addr 172.21.15.16:0/2190936239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.514+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.514+0000 7f7ba0c00700 1 -- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 msgr2=0x563bf45b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.515+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.515+0000 7f7ba0c00700 1 -- 172.21.15.16:0/2190936239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf4429e40 msgr2=0x563bf45b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.515+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf4429e40 0x563bf45b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.516+0000 7f7ba0c00700 1 -- 172.21.15.16:0/2190936239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf44168d0 con 0x563bf45159f0 2024-05-27T07:03:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.516+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.518+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0128d0 tx=0x7f7b8c0129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81895ffebdda5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.518+0000 7f7b7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:03:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.519+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c027020 con 0x563bf45159f0 2024-05-27T07:03:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.519+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c012a80 con 0x563bf45159f0 2024-05-27T07:03:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.519+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7b8c0054b0 con 0x563bf45159f0 2024-05-27T07:03:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.520+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c032050 con 0x563bf45159f0 2024-05-27T07:03:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.520+0000 7f7bb21c1b80 10 client.35540 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.520+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bf4497060 con 0x563bf45159f0 2024-05-27T07:03:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.521+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b8c012a80 con 0x563bf45159f0 2024-05-27T07:03:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7b8c004be0 con 0x563bf45159f0 2024-05-27T07:03:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 10 client.35540 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 10 client.35540 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:03:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 20 client.35540 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf45d3b20 0x563bf45d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 4 client.35540 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.522+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bf44d8d90 con 0x563bf45d3b20 2024-05-27T07:03:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.524+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf45d3b20 0x563bf45d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.526+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf45d3b20 0x563bf45d5f40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59237826eabe124e server_cookie=efe403410596fbfe in_seq=0 out_seq=0 2024-05-27T07:03:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.526+0000 7f7b7ea00700 10 client.35540 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:03:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.530+0000 7f7b7ea00700 1 -- 172.21.15.16:0/2190936239 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563bf44d8d90 con 0x563bf45d3b20 2024-05-27T07:03:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.530+0000 7f7b7ea00700 10 client.35540 handle_command_reply: tid=0 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.530+0000 7f7bb21c1b80 1 client.35540 shutdown 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.530+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 msgr2=0x563bf45b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.530+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0128d0 tx=0x7f7b8c0129b0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 shutdown_connections 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf45d3b20 0x563bf45d5f40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf4429e40 0x563bf45b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2190936239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45159f0 0x563bf45b2c80 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0128d0 tx=0x7f7b8c0129b0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 >> 172.21.15.16:0/2190936239 conn(0x563bf44c3390 msgr2=0x563bf45a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.531+0000 7f7b7ea00700 0 client.35540 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:03:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.533+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 shutdown_connections 2024-05-27T07:03:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.533+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2190936239 wait complete. 2024-05-27T07:03:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.533+0000 7f7bb21c1b80 20 client.35540 trim_cache size 0 max 16384 2024-05-27T07:03:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 Processor -- start 2024-05-27T07:03:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 -- start start 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45adef0 0x563bf45b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8000ed20 con 0x563bf45a8e70 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b80010860 con 0x563bf4429e40 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.561+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8000f3f0 con 0x563bf45adef0 2024-05-27T07:03:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.562+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.562+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:33960/0 (socket says 172.21.15.16:33960) 2024-05-27T07:03:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.562+0000 7f7b9be00700 1 -- 172.21.15.16:0/2624903853 learned_addr learned my addr 172.21.15.16:0/2624903853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45adef0 0x563bf45b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 -- 172.21.15.16:0/2624903853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45adef0 msgr2=0x563bf45b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45adef0 0x563bf45b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 -- 172.21.15.16:0/2624903853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 msgr2=0x563bf45b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.563+0000 7f7b9be00700 1 -- 172.21.15.16:0/2624903853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf45a0240 con 0x563bf4429e40 2024-05-27T07:03:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.564+0000 7f7b9be00700 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563bf45aaff0 tx=0x7f7b9000b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d028afd101522c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.565+0000 7f7b89600700 1 -- 172.21.15.16:0/2624903853 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9001b520 con 0x563bf4429e40 2024-05-27T07:03:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.567+0000 7f7b89600700 1 -- 172.21.15.16:0/2624903853 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b9001bb50 con 0x563bf4429e40 2024-05-27T07:03:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.567+0000 7f7b89600700 1 -- 172.21.15.16:0/2624903853 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011b50 con 0x563bf4429e40 2024-05-27T07:03:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.568+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2624903853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 msgr2=0x563bf45a8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.568+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563bf45aaff0 tx=0x7f7b9000b470 comp rx=0 tx=0).stop 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2624903853 shutdown_connections 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45adef0 0x563bf45b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf4429e40 0x563bf45a8420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/2624903853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45a8e70 0x563bf45b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2624903853 >> 172.21.15.16:0/2624903853 conn(0x563bf44c3390 msgr2=0x563bf45b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2624903853 shutdown_connections 2024-05-27T07:03:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.569+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/2624903853 wait complete. 2024-05-27T07:03:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.570+0000 7f7bb21c1b80 1 -- start start 2024-05-27T07:03:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 Processor -- start 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 0x563bf45b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b80010190 con 0x563bf45b5e60 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8000fac0 con 0x563bf45adef0 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8000ccc0 con 0x563bf45a8e70 2024-05-27T07:03:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.571+0000 7f7bb21c1b80 10 client.0 _ll_register_callbacks cb 0x563bf4470c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.572+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.573+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 0x563bf45b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.573+0000 7f7ba0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:33968/0 (socket says 172.21.15.16:33968) 2024-05-27T07:03:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.573+0000 7f7ba0c00700 1 -- 172.21.15.16:0/1667794670 learned_addr learned my addr 172.21.15.16:0/1667794670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7b9be00700 1 -- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 msgr2=0x563bf45baeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7b9be00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45baeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7b9be00700 1 -- 172.21.15.16:0/1667794670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 msgr2=0x563bf45bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7b9be00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7b9be00700 1 -- 172.21.15.16:0/1667794670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf45bba60 con 0x563bf45b5e60 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.574+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.575+0000 7f7b9be00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 0x563bf45b6270 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x563bf45b5850 tx=0x7f7b9000bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9881effb935f792d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.575+0000 7f7b7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.575+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011440 con 0x563bf45b5e60 2024-05-27T07:03:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.576+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7b90004020 con 0x563bf45b5e60 2024-05-27T07:03:18.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.576+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9001bb50 con 0x563bf45b5e60 2024-05-27T07:03:18.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.576+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b90004220 con 0x563bf45b5e60 2024-05-27T07:03:18.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.576+0000 7f7bb21c1b80 10 client.15813 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:18.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.577+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bf463ed30 con 0x563bf45b5e60 2024-05-27T07:03:18.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.577+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b9001bb50 con 0x563bf45b5e60 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7b900117d0 con 0x563bf45b5e60 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 10 client.15813 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 20 client.15813 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf463f290 0x563bf46416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 4 client.15813 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:03:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.578+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bf44d2570 con 0x563bf463f290 2024-05-27T07:03:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.579+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf463f290 0x563bf46416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.580+0000 7f7ba0c00700 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf463f290 0x563bf46416b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16ba6d30f7c3fc21 server_cookie=722e325ea5263f5 in_seq=0 out_seq=0 2024-05-27T07:03:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.580+0000 7f7b7ea00700 10 client.15813 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:03:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.581+0000 7f7b7ea00700 1 -- 172.21.15.16:0/1667794670 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bf44d2570 con 0x563bf463f290 2024-05-27T07:03:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.581+0000 7f7b7ea00700 10 client.15813 handle_command_reply: tid=0 2024-05-27T07:03:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.581+0000 7f7bb21c1b80 1 client.15813 shutdown 2024-05-27T07:03:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.582+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 msgr2=0x563bf45b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.582+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 0x563bf45b6270 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x563bf45b5850 tx=0x7f7b9000bae0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 shutdown_connections 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bf463f290 0x563bf46416b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bf45b5e60 0x563bf45b6270 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf45a8e70 0x563bf45baeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7b7ea00700 0 client.15813 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 --2- 172.21.15.16:0/1667794670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bf45adef0 0x563bf45bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 >> 172.21.15.16:0/1667794670 conn(0x563bf44c3390 msgr2=0x563bf45b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 shutdown_connections 2024-05-27T07:03:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.583+0000 7f7bb21c1b80 1 -- 172.21.15.16:0/1667794670 wait complete. 2024-05-27T07:03:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.584+0000 7f7bb21c1b80 20 client.15813 trim_cache size 0 max 16384 2024-05-27T07:03:18.578 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:03:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.586+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 msgr2=0x7f7b840b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.586+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 0x7f7b840b5000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b940a6e60 tx=0x7f7b94009040 comp rx=0 tx=0).stop 2024-05-27T07:03:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.586+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 msgr2=0x7f7b9c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.586+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c116a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400b0a0 tx=0x7f7b94004d30 comp rx=0 tx=0).stop 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 shutdown_connections 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7b840b2af0 0x7f7b840b5000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c1192b0 0x7f7b9c116a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b9c10abb0 0x7f7b9c116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 --2- 172.21.15.16:0/1696097927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c10b500 0x7f7b9c116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.587+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 >> 172.21.15.16:0/1696097927 conn(0x7f7b9c06f4c0 msgr2=0x7f7b9c1198c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.588+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 shutdown_connections 2024-05-27T07:03:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:18.588+0000 7f7ba3600700 1 -- 172.21.15.16:0/1696097927 wait complete. 2024-05-27T07:03:19.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:03:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:19 smithi164 ceph-mon[35327]: pgmap v75: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-27T07:03:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:19 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:03:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:19 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:03:19.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:19 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:03:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:19 smithi016 ceph-mon[31415]: pgmap v75: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-27T07:03:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:19 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:03:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:19 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:03:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:19 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:03:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:19 smithi069 ceph-mon[35276]: pgmap v75: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-27T07:03:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:19 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:03:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:19 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:03:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:19 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:03:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:21.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:21 smithi164 ceph-mon[35327]: pgmap v76: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 124 MiB/s wr, 78 op/s 2024-05-27T07:03:21.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:21 smithi016 ceph-mon[31415]: pgmap v76: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 124 MiB/s wr, 78 op/s 2024-05-27T07:03:21.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:21 smithi069 ceph-mon[35276]: pgmap v76: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 124 MiB/s wr, 78 op/s 2024-05-27T07:03:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.697+0000 7fde89000700 1 -- 172.21.15.16:0/2112349541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde841045e0 msgr2=0x7fde841049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.697+0000 7fde89000700 1 --2- 172.21.15.16:0/2112349541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde841045e0 0x7fde841049f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b0d0 tx=0x7fde7c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.698+0000 7fde89000700 1 -- 172.21.15.16:0/2112349541 shutdown_connections 2024-05-27T07:03:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.698+0000 7fde89000700 1 --2- 172.21.15.16:0/2112349541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde84109860 0x7fde841111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.698+0000 7fde89000700 1 --2- 172.21.15.16:0/2112349541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde841045e0 0x7fde841049f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.698+0000 7fde89000700 1 --2- 172.21.15.16:0/2112349541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84108f10 0x7fde84109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.698+0000 7fde89000700 1 -- 172.21.15.16:0/2112349541 >> 172.21.15.16:0/2112349541 conn(0x7fde84100030 msgr2=0x7fde84102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.700+0000 7fde89000700 1 -- 172.21.15.16:0/2112349541 shutdown_connections 2024-05-27T07:03:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.700+0000 7fde89000700 1 -- 172.21.15.16:0/2112349541 wait complete. 2024-05-27T07:03:22.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.702+0000 7fde89000700 1 Processor -- start 2024-05-27T07:03:22.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.702+0000 7fde89000700 1 -- start start 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 0x7fde8419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 0x7fde8419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84109860 0x7fde841a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde840775b0 con 0x7fde84108f10 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde84077430 con 0x7fde841045e0 2024-05-27T07:03:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde89000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde84077730 con 0x7fde84109860 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde81e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 0x7fde8419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 0x7fde8419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde82800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84109860 0x7fde841a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.703+0000 7fde81e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 0x7fde8419cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43234/0 (socket says 172.21.15.16:43234) 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.704+0000 7fde81e00700 1 -- 172.21.15.16:0/9931910 learned_addr learned my addr 172.21.15.16:0/9931910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.704+0000 7fde81400700 1 -- 172.21.15.16:0/9931910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84109860 msgr2=0x7fde841a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.704+0000 7fde81400700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84109860 0x7fde841a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.704+0000 7fde81400700 1 -- 172.21.15.16:0/9931910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 msgr2=0x7fde8419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.704+0000 7fde81400700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 0x7fde8419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.705+0000 7fde81400700 1 -- 172.21.15.16:0/9931910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde841a1e50 con 0x7fde84108f10 2024-05-27T07:03:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.705+0000 7fde81400700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 0x7fde8419d520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e9c0 tx=0x7fde6c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.706+0000 7fde7aa00700 1 -- 172.21.15.16:0/9931910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00c9c0 con 0x7fde84108f10 2024-05-27T07:03:22.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.706+0000 7fde7aa00700 1 -- 172.21.15.16:0/9931910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde6c010430 con 0x7fde84108f10 2024-05-27T07:03:22.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.706+0000 7fde7aa00700 1 -- 172.21.15.16:0/9931910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c018680 con 0x7fde84108f10 2024-05-27T07:03:22.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.706+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde841a2140 con 0x7fde84108f10 2024-05-27T07:03:22.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.706+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde841a9a60 con 0x7fde84108f10 2024-05-27T07:03:22.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.713+0000 7fde97c08b80 1 Processor -- start 2024-05-27T07:03:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.714+0000 7fde7aa00700 1 -- 172.21.15.16:0/9931910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fde6c018820 con 0x7fde84108f10 2024-05-27T07:03:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.714+0000 7fde97c08b80 1 -- start start 2024-05-27T07:03:22.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40d2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9e3febe40 con 0x55e9e3f3ee40 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9e3fdb9e0 con 0x55e9e402a9f0 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde97c08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9e3fdfa20 con 0x55e9e40bdd00 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde82800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde82800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43250/0 (socket says 172.21.15.16:43250) 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.715+0000 7fde82800700 1 -- 172.21.15.16:0/400835566 learned_addr learned my addr 172.21.15.16:0/400835566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.716+0000 7fde81e00700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.716+0000 7fde81400700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40d2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.716+0000 7fde82800700 1 -- 172.21.15.16:0/400835566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 msgr2=0x55e9e40d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde82800700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40d2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde82800700 1 -- 172.21.15.16:0/400835566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 msgr2=0x55e9e40bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde82800700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde82800700 1 -- 172.21.15.16:0/400835566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e3fa4420 con 0x55e9e402a9f0 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde81e00700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde81400700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde82800700 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde74009cb0 tx=0x7fde74011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f32e3ccef491818 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.717+0000 7fde7aa00700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 0x7fde680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.718+0000 7fde7aa00700 1 -- 172.21.15.16:0/9931910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fde6c0e2ab0 con 0x7fde84108f10 2024-05-27T07:03:22.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.718+0000 7fde73400700 1 -- 172.21.15.16:0/400835566 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74011e30 con 0x55e9e402a9f0 2024-05-27T07:03:22.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.718+0000 7fde81e00700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 0x7fde680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.718+0000 7fde73400700 1 -- 172.21.15.16:0/400835566 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde740044c0 con 0x55e9e402a9f0 2024-05-27T07:03:22.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.718+0000 7fde73400700 1 -- 172.21.15.16:0/400835566 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74019420 con 0x55e9e402a9f0 2024-05-27T07:03:22.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.720+0000 7fde97c08b80 1 -- 172.21.15.16:0/400835566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 msgr2=0x55e9e40bd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.720+0000 7fde97c08b80 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde74009cb0 tx=0x7fde74011440 comp rx=0 tx=0).stop 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.720+0000 7fde81e00700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 0x7fde680b4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009e60 tx=0x7fde7c00b040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 -- 172.21.15.16:0/400835566 shutdown_connections 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40bd230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 --2- 172.21.15.16:0/400835566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 -- 172.21.15.16:0/400835566 >> 172.21.15.16:0/400835566 conn(0x55e9e3fd8390 msgr2=0x55e9e40b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.721+0000 7fde97c08b80 1 -- 172.21.15.16:0/400835566 shutdown_connections 2024-05-27T07:03:22.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.722+0000 7fde97c08b80 1 -- 172.21.15.16:0/400835566 wait complete. 2024-05-27T07:03:22.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.722+0000 7fde97c08b80 1 -- start start 2024-05-27T07:03:22.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.723+0000 7fde97c08b80 1 Processor -- start 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.723+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 0x55e9e40cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.723+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.723+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 0x55e9e40d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde97c08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9e3ee1fa0 con 0x55e9e40bdd00 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde97c08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9e3f7c760 con 0x55e9e402a9f0 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde97c08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9e40d33a0 con 0x55e9e3f3ee40 2024-05-27T07:03:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde97c08b80 10 client.0 _ll_register_callbacks cb 0x55e9e3f85c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde81e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 0x55e9e40cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.724+0000 7fde81e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 0x55e9e40cfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2024-05-27T07:03:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde81e00700 1 -- 172.21.15.16:0/183220364 learned_addr learned my addr 172.21.15.16:0/183220364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde81400700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 0x55e9e40d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 -- 172.21.15.16:0/183220364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 msgr2=0x55e9e40cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 0x55e9e40cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 -- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 msgr2=0x55e9e40d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.725+0000 7fde82800700 1 -- 172.21.15.16:0/183220364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e3f2b8d0 con 0x55e9e40bdd00 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.726+0000 7fde82800700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 0x55e9e40d0a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fde74009cb0 tx=0x7fde74004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51808ce857450ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.726+0000 7fde67e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74005240 con 0x55e9e40bdd00 2024-05-27T07:03:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fde740053e0 con 0x55e9e40bdd00 2024-05-27T07:03:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde740044c0 con 0x55e9e40bdd00 2024-05-27T07:03:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fde74005620 con 0x55e9e40bdd00 2024-05-27T07:03:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde97c08b80 10 client.15825 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.727+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e9e3fac060 con 0x55e9e40bdd00 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.728+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde740044c0 con 0x55e9e40bdd00 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.728+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fde74004c90 con 0x55e9e40bdd00 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 10 client.15825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 20 client.15825 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e40e01f0 0x55e9e40e2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 4 client.15825 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.730+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e3fedd90 con 0x55e9e40e01f0 2024-05-27T07:03:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.731+0000 7fde81e00700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e40e01f0 0x55e9e40e2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.732+0000 7fde81e00700 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e40e01f0 0x55e9e40e2610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1cf866d77ca5c44 server_cookie=8b46aaad6bc5f5d in_seq=0 out_seq=0 2024-05-27T07:03:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.732+0000 7fde67e00700 10 client.15825 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:03:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde67e00700 1 -- 172.21.15.16:0/183220364 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9e3fedd90 con 0x55e9e40e01f0 2024-05-27T07:03:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde67e00700 10 client.15825 handle_command_reply: tid=0 2024-05-27T07:03:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 client.15825 shutdown 2024-05-27T07:03:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 msgr2=0x55e9e40d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 0x55e9e40d0a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fde74009cb0 tx=0x7fde74004330 comp rx=0 tx=0).stop 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 shutdown_connections 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e40bdd00 0x55e9e40d0a70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e3f3ee40 0x55e9e40cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e40e01f0 0x55e9e40e2610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 --2- 172.21.15.16:0/183220364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e402a9f0 0x55e9e40d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.735+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 >> 172.21.15.16:0/183220364 conn(0x55e9e3fd8390 msgr2=0x55e9e40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.736+0000 7fde67e00700 0 client.15825 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:03:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.736+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 shutdown_connections 2024-05-27T07:03:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.736+0000 7fde97c08b80 1 -- 172.21.15.16:0/183220364 wait complete. 2024-05-27T07:03:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.736+0000 7fde97c08b80 20 client.15825 trim_cache size 0 max 16384 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.761+0000 7fde97c08b80 1 Processor -- start 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.761+0000 7fde97c08b80 1 -- start start 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde841ae460 con 0x55e9e402a9f0 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde841affa0 con 0x55e9e3f3ee40 2024-05-27T07:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde97c08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde841aeb30 con 0x55e9e40bdd00 2024-05-27T07:03:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57680/0 (socket says 172.21.15.16:57680) 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde81400700 1 -- 172.21.15.16:0/2947342640 learned_addr learned my addr 172.21.15.16:0/2947342640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.762+0000 7fde81e00700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde82800700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 -- 172.21.15.16:0/2947342640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 msgr2=0x55e9e40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 -- 172.21.15.16:0/2947342640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 msgr2=0x55e9e40bd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 -- 172.21.15.16:0/2947342640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e40d0de0 con 0x55e9e402a9f0 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81e00700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.763+0000 7fde81400700 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0dfed0 tx=0x7fde6c0a58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6377b1ff80b0e0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.764+0000 7fde72a00700 1 -- 172.21.15.16:0/2947342640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0aa100 con 0x55e9e402a9f0 2024-05-27T07:03:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.764+0000 7fde72a00700 1 -- 172.21.15.16:0/2947342640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde6c0a9a00 con 0x55e9e402a9f0 2024-05-27T07:03:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.764+0000 7fde72a00700 1 -- 172.21.15.16:0/2947342640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a9ec0 con 0x55e9e402a9f0 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.765+0000 7fde97c08b80 1 -- 172.21.15.16:0/2947342640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 msgr2=0x55e9e40cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.765+0000 7fde97c08b80 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0dfed0 tx=0x7fde6c0a58c0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 -- 172.21.15.16:0/2947342640 shutdown_connections 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e40ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 --2- 172.21.15.16:0/2947342640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40cb820 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 -- 172.21.15.16:0/2947342640 >> 172.21.15.16:0/2947342640 conn(0x55e9e3fd8390 msgr2=0x55e9e40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 -- 172.21.15.16:0/2947342640 shutdown_connections 2024-05-27T07:03:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.766+0000 7fde97c08b80 1 -- 172.21.15.16:0/2947342640 wait complete. 2024-05-27T07:03:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.767+0000 7fde97c08b80 1 -- start start 2024-05-27T07:03:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.767+0000 7fde97c08b80 1 Processor -- start 2024-05-27T07:03:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e4148a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde841af8d0 con 0x55e9e402a9f0 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde841af200 con 0x55e9e3f3ee40 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.768+0000 7fde97c08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde841ac400 con 0x55e9e40bdd00 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.769+0000 7fde97c08b80 10 client.0 _ll_register_callbacks cb 0x55e9e3f85c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.769+0000 7fde81e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.769+0000 7fde81e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43288/0 (socket says 172.21.15.16:43288) 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.769+0000 7fde81e00700 1 -- 172.21.15.16:0/1824381562 learned_addr learned my addr 172.21.15.16:0/1824381562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.769+0000 7fde81400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde82800700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e4148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81400700 1 -- 172.21.15.16:0/1824381562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 msgr2=0x55e9e4148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81400700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e4148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81400700 1 -- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 msgr2=0x55e9e40e07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81400700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81400700 1 -- 172.21.15.16:0/1824381562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e4148f80 con 0x55e9e402a9f0 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.770+0000 7fde81e00700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.771+0000 7fde81400700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40e0d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55e9e40ba0b0 tx=0x7fde6c0a9fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915517798d9e563d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.771+0000 7fde67e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0b2860 con 0x55e9e402a9f0 2024-05-27T07:03:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fde6c0a5ff0 con 0x55e9e402a9f0 2024-05-27T07:03:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c0a9a00 con 0x55e9e402a9f0 2024-05-27T07:03:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fde6c0aa100 con 0x55e9e402a9f0 2024-05-27T07:03:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde97c08b80 10 client.15831 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.772+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e9e414a0b0 con 0x55e9e402a9f0 2024-05-27T07:03:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde6c0a9a00 con 0x55e9e402a9f0 2024-05-27T07:03:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fde6c0c5020 con 0x55e9e402a9f0 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 10 client.15831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 20 client.15831 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e414a710 0x55e9e414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 4 client.15831 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:03:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.773+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9e3fe7570 con 0x55e9e414a710 2024-05-27T07:03:22.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.774+0000 7fde81e00700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e414a710 0x55e9e414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:22.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.780+0000 7fde67e00700 10 client.15831 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:03:22.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.780+0000 7fde81e00700 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e414a710 0x55e9e414cb30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db7118524e26f2a server_cookie=ff6a1c9d2154b628 in_seq=0 out_seq=0 2024-05-27T07:03:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.781+0000 7fde67e00700 1 -- 172.21.15.16:0/1824381562 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9e3fe7570 con 0x55e9e414a710 2024-05-27T07:03:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.781+0000 7fde67e00700 10 client.15831 handle_command_reply: tid=0 2024-05-27T07:03:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.781+0000 7fde97c08b80 1 client.15831 shutdown 2024-05-27T07:03:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 msgr2=0x55e9e40e0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40e0d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55e9e40ba0b0 tx=0x7fde6c0a9fc0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 shutdown_connections 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9e40bdd00 0x55e9e4148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e9e414a710 0x55e9e414cb30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9e3f3ee40 0x55e9e40e07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 --2- 172.21.15.16:0/1824381562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e402a9f0 0x55e9e40e0d40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde67e00700 0 client.15831 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.782+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 >> 172.21.15.16:0/1824381562 conn(0x55e9e3fd8390 msgr2=0x55e9e40bef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.783+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 shutdown_connections 2024-05-27T07:03:22.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.783+0000 7fde97c08b80 1 -- 172.21.15.16:0/1824381562 wait complete. 2024-05-27T07:03:22.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.783+0000 7fde97c08b80 20 client.15831 trim_cache size 0 max 16384 2024-05-27T07:03:22.776 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:03:22.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.784+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 msgr2=0x7fde680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.784+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 0x7fde680b4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009e60 tx=0x7fde7c00b040 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.784+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 msgr2=0x7fde8419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.784+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 0x7fde8419d520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e9c0 tx=0x7fde6c00ecd0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 shutdown_connections 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde680b2a70 0x7fde680b4f80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde84109860 0x7fde841a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841045e0 0x7fde8419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 --2- 172.21.15.16:0/9931910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84108f10 0x7fde8419d520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 >> 172.21.15.16:0/9931910 conn(0x7fde84100030 msgr2=0x7fde84102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 shutdown_connections 2024-05-27T07:03:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:22.785+0000 7fde89000700 1 -- 172.21.15.16:0/9931910 wait complete. 2024-05-27T07:03:23.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:23 smithi016 ceph-mon[31415]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:23.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:03:23.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:23 smithi164 ceph-mon[35327]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:23 smithi069 ceph-mon[35276]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:25.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:25 smithi016 ceph-mon[31415]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:25.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:25 smithi164 ceph-mon[35327]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:25.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:25 smithi069 ceph-mon[35276]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 -- 172.21.15.16:0/4215074055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698108730 msgr2=0x7f1698110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 --2- 172.21.15.16:0/4215074055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698108730 0x7f1698110090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f169000b0a0 tx=0x7f169000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 -- 172.21.15.16:0/4215074055 shutdown_connections 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 --2- 172.21.15.16:0/4215074055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698108730 0x7f1698110090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 --2- 172.21.15.16:0/4215074055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16981034b0 0x7f16981038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 --2- 172.21.15.16:0/4215074055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1698107de0 0x7f16981081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 -- 172.21.15.16:0/4215074055 >> 172.21.15.16:0/4215074055 conn(0x7f16980fedb0 msgr2=0x7f16981011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 -- 172.21.15.16:0/4215074055 shutdown_connections 2024-05-27T07:03:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.075+0000 7f169d400700 1 -- 172.21.15.16:0/4215074055 wait complete. 2024-05-27T07:03:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.078+0000 7f169d400700 1 Processor -- start 2024-05-27T07:03:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.078+0000 7f169d400700 1 -- start start 2024-05-27T07:03:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.079+0000 7f169d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.079+0000 7f169d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 0x7f169819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f169d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698108730 0x7f16981a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f169d400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16981143b0 con 0x7f1698107de0 2024-05-27T07:03:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f169d400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1698114230 con 0x7f16981034b0 2024-05-27T07:03:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f1696800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f1696800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43316/0 (socket says 172.21.15.16:43316) 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f1696800700 1 -- 172.21.15.16:0/771714636 learned_addr learned my addr 172.21.15.16:0/771714636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1698114530 con 0x7f1698108730 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.080+0000 7f1697200700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698108730 0x7f16981a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 0x7f169819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 -- 172.21.15.16:0/771714636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698108730 msgr2=0x7f16981a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698108730 0x7f16981a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 -- 172.21.15.16:0/771714636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 msgr2=0x7f169819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.081+0000 7f1695e00700 1 -- 172.21.15.16:0/771714636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16981a1df0 con 0x7f1698107de0 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.082+0000 7f1696800700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.082+0000 7f1695e00700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 0x7f169819d4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f168400e9c0 tx=0x7f168400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.082+0000 7f168f400700 1 -- 172.21.15.16:0/771714636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400c9c0 con 0x7f1698107de0 2024-05-27T07:03:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.083+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16981a2080 con 0x7f1698107de0 2024-05-27T07:03:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.083+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a9a00 con 0x7f1698107de0 2024-05-27T07:03:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.082+0000 7f168f400700 1 -- 172.21.15.16:0/771714636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1684010430 con 0x7f1698107de0 2024-05-27T07:03:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.083+0000 7f168f400700 1 -- 172.21.15.16:0/771714636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684018680 con 0x7f1698107de0 2024-05-27T07:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.089+0000 7f16ac036b80 1 Processor -- start 2024-05-27T07:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.089+0000 7f16ac036b80 1 -- start start 2024-05-27T07:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e628e40 0x55b42e7b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b42e6d5e40 con 0x55b42e7b05c0 2024-05-27T07:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b42e6c59e0 con 0x55b42e7b64e0 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f16ac036b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b42e6c9a20 con 0x55b42e628e40 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f168f400700 1 -- 172.21.15.16:0/771714636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1684018820 con 0x7f1698107de0 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.090+0000 7f1695e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.091+0000 7f1695e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57714/0 (socket says 172.21.15.16:57714) 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.091+0000 7f1695e00700 1 -- 172.21.15.16:0/1930630116 learned_addr learned my addr 172.21.15.16:0/1930630116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.091+0000 7f1696800700 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f1695e00700 1 -- 172.21.15.16:0/1930630116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e628e40 msgr2=0x55b42e7b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f1695e00700 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e628e40 0x55b42e7b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f1695e00700 1 -- 172.21.15.16:0/1930630116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 msgr2=0x55b42e7b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f1695e00700 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f1695e00700 1 -- 172.21.15.16:0/1930630116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42e68e420 con 0x55b42e7b05c0 2024-05-27T07:03:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.092+0000 7f168f400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 0x7f167c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f168f400700 1 -- 172.21.15.16:0/771714636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f16840e28c0 con 0x7f1698107de0 2024-05-27T07:03:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f1695e00700 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f16840dfbf0 tx=0x7f16840e8ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cc69d665431c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f1697200700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 0x7f167c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f1683e00700 1 -- 172.21.15.16:0/1930630116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840a56d0 con 0x55b42e7b05c0 2024-05-27T07:03:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f1683e00700 1 -- 172.21.15.16:0/1930630116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16840a9320 con 0x55b42e7b05c0 2024-05-27T07:03:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.093+0000 7f1683e00700 1 -- 172.21.15.16:0/1930630116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840a8f80 con 0x55b42e7b05c0 2024-05-27T07:03:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.095+0000 7f1697200700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 0x7f167c0b4fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1690009900 tx=0x7f1690006000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.095+0000 7f16ac036b80 1 -- 172.21.15.16:0/1930630116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 msgr2=0x55b42e7b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.095+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f16840dfbf0 tx=0x7f16840e8ef0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.096+0000 7f16ac036b80 1 -- 172.21.15.16:0/1930630116 shutdown_connections 2024-05-27T07:03:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.096+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7b05c0 0x55b42e7b09d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.096+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e628e40 0x55b42e7b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.096+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1930630116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.096+0000 7f16ac036b80 1 -- 172.21.15.16:0/1930630116 >> 172.21.15.16:0/1930630116 conn(0x55b42e688fc0 msgr2=0x55b42e7a28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.097+0000 7f16ac036b80 1 -- 172.21.15.16:0/1930630116 shutdown_connections 2024-05-27T07:03:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.097+0000 7f16ac036b80 1 -- 172.21.15.16:0/1930630116 wait complete. 2024-05-27T07:03:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.098+0000 7f16ac036b80 1 -- start start 2024-05-27T07:03:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.098+0000 7f16ac036b80 1 Processor -- start 2024-05-27T07:03:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.098+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.098+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7ba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f16ac036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b42e5cbfa0 con 0x55b42e628e40 2024-05-27T07:03:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f16ac036b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b42e666760 con 0x55b42e7b64e0 2024-05-27T07:03:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f16ac036b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b42e7bcfc0 con 0x55b42e7b05c0 2024-05-27T07:03:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f16ac036b80 10 client.0 _ll_register_callbacks cb 0x55b42e66fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f1696800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f1696800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57724/0 (socket says 172.21.15.16:57724) 2024-05-27T07:03:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.099+0000 7f1696800700 1 -- 172.21.15.16:0/3281260165 learned_addr learned my addr 172.21.15.16:0/3281260165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.100+0000 7f1695e00700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.100+0000 7f1697200700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1696800700 1 -- 172.21.15.16:0/3281260165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 msgr2=0x55b42e7ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1696800700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1696800700 1 -- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 msgr2=0x55b42e7ba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1696800700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7ba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1696800700 1 -- 172.21.15.16:0/3281260165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b42e6158d0 con 0x55b42e628e40 2024-05-27T07:03:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.101+0000 7f1695e00700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.102+0000 7f1696800700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1688012910 tx=0x7f1688012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ccf7b35802ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.102+0000 7f1680c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.102+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168801f060 con 0x55b42e628e40 2024-05-27T07:03:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.102+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1688004460 con 0x55b42e628e40 2024-05-27T07:03:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.102+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16840a9320 con 0x55b42e628e40 2024-05-27T07:03:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.103+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1688003680 con 0x55b42e628e40 2024-05-27T07:03:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.103+0000 7f16ac036b80 10 client.15843 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.103+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b42e696060 con 0x55b42e628e40 2024-05-27T07:03:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.104+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16840a9320 con 0x55b42e628e40 2024-05-27T07:03:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.104+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1688019030 con 0x55b42e628e40 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 10 client.15843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 20 client.15843 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e7c9ef0 0x55b42e7cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 4 client.15843 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.105+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b42e6d7d90 con 0x55b42e7c9ef0 2024-05-27T07:03:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.106+0000 7f1695e00700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e7c9ef0 0x55b42e7cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.107+0000 7f1695e00700 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e7c9ef0 0x55b42e7cc310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ff93a83e6fd2aaa server_cookie=19c95997fd0d0a0a in_seq=0 out_seq=0 2024-05-27T07:03:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.107+0000 7f1680c00700 10 client.15843 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:03:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.110+0000 7f1680c00700 1 -- 172.21.15.16:0/3281260165 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b42e6d7d90 con 0x55b42e7c9ef0 2024-05-27T07:03:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.110+0000 7f1680c00700 10 client.15843 handle_command_reply: tid=0 2024-05-27T07:03:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.110+0000 7f16ac036b80 1 client.15843 shutdown 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 msgr2=0x55b42e7b9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1688012910 tx=0x7f1688012c20 comp rx=0 tx=0).stop 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 shutdown_connections 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e7c9ef0 0x55b42e7cc310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f1680c00700 0 client.15843 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7b9d10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 --2- 172.21.15.16:0/3281260165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.111+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 >> 172.21.15.16:0/3281260165 conn(0x55b42e688fc0 msgr2=0x55b42e7a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.112+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 shutdown_connections 2024-05-27T07:03:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.112+0000 7f16ac036b80 1 -- 172.21.15.16:0/3281260165 wait complete. 2024-05-27T07:03:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.112+0000 7f16ac036b80 20 client.15843 trim_cache size 0 max 16384 2024-05-27T07:03:27.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.140+0000 7f16ac036b80 1 Processor -- start 2024-05-27T07:03:27.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.140+0000 7f16ac036b80 1 -- start start 2024-05-27T07:03:27.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f167800ee00 con 0x55b42e628e40 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1678010940 con 0x55b42e7b64e0 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f16ac036b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f167800f4d0 con 0x55b42e7b05c0 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.141+0000 7f1696800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.142+0000 7f1696800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57728/0 (socket says 172.21.15.16:57728) 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.142+0000 7f1696800700 1 -- 172.21.15.16:0/1306266826 learned_addr learned my addr 172.21.15.16:0/1306266826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.142+0000 7f1695e00700 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.142+0000 7f1697200700 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.143+0000 7f1696800700 1 -- 172.21.15.16:0/1306266826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 msgr2=0x55b42e7a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.143+0000 7f1696800700 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.143+0000 7f1696800700 1 -- 172.21.15.16:0/1306266826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 msgr2=0x55b42e7b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.143+0000 7f1696800700 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.143+0000 7f1696800700 1 -- 172.21.15.16:0/1306266826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42e7b9030 con 0x55b42e628e40 2024-05-27T07:03:27.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.144+0000 7f1696800700 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1688010bf0 tx=0x7f168801a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3472f219fb1e029d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.144+0000 7f1683400700 1 -- 172.21.15.16:0/1306266826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688029080 con 0x55b42e628e40 2024-05-27T07:03:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.145+0000 7f1683400700 1 -- 172.21.15.16:0/1306266826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1688010e60 con 0x55b42e628e40 2024-05-27T07:03:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.145+0000 7f1683400700 1 -- 172.21.15.16:0/1306266826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688003df0 con 0x55b42e628e40 2024-05-27T07:03:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 -- 172.21.15.16:0/1306266826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 msgr2=0x55b42e7c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1688010bf0 tx=0x7f168801a3f0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 -- 172.21.15.16:0/1306266826 shutdown_connections 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b64e0 0x55b42e7b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e628e40 0x55b42e7c60f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 --2- 172.21.15.16:0/1306266826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b05c0 0x55b42e7a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 -- 172.21.15.16:0/1306266826 >> 172.21.15.16:0/1306266826 conn(0x55b42e688fc0 msgr2=0x55b42e7a6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.146+0000 7f16ac036b80 1 -- 172.21.15.16:0/1306266826 shutdown_connections 2024-05-27T07:03:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.147+0000 7f16ac036b80 1 -- 172.21.15.16:0/1306266826 wait complete. 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.147+0000 7f16ac036b80 1 -- start start 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.147+0000 7f16ac036b80 1 Processor -- start 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b05c0 0x55b42e7c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b64e0 0x55b42e7d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1678010270 con 0x55b42e7c9ef0 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f167800fba0 con 0x55b42e7b05c0 2024-05-27T07:03:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f167800cda0 con 0x55b42e7b64e0 2024-05-27T07:03:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.148+0000 7f16ac036b80 10 client.0 _ll_register_callbacks cb 0x55b42e66fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57734/0 (socket says 172.21.15.16:57734) 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 -- 172.21.15.16:0/283393773 learned_addr learned my addr 172.21.15.16:0/283393773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1695e00700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b64e0 0x55b42e7d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1696800700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b05c0 0x55b42e7c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 -- 172.21.15.16:0/283393773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b64e0 msgr2=0x55b42e7d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b64e0 0x55b42e7d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 -- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b05c0 msgr2=0x55b42e7c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b05c0 0x55b42e7c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.149+0000 7f1697200700 1 -- 172.21.15.16:0/283393773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b42e7d1aa0 con 0x55b42e7c9ef0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.150+0000 7f1697200700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f169000b0a0 tx=0x7f169000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037ccadbde05220 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.150+0000 7f1680c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.150+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690007b10 con 0x55b42e7c9ef0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.151+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1690007cb0 con 0x55b42e7c9ef0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.151+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688010e60 con 0x55b42e7c9ef0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.151+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1690005040 con 0x55b42e7c9ef0 2024-05-27T07:03:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.151+0000 7f16ac036b80 10 client.15849 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:27.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.151+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b42e834470 con 0x55b42e7c9ef0 2024-05-27T07:03:27.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.152+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1688010e60 con 0x55b42e7c9ef0 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f169003d020 con 0x55b42e7c9ef0 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 10 client.15849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 20 client.15849 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e834b80 0x55b42e836fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 4 client.15849 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.153+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b42e6d1570 con 0x55b42e834b80 2024-05-27T07:03:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.154+0000 7f1696800700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e834b80 0x55b42e836fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.155+0000 7f1696800700 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e834b80 0x55b42e836fa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=907d5a09e952ccf server_cookie=735eec23ec486cfe in_seq=0 out_seq=0 2024-05-27T07:03:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.155+0000 7f1680c00700 10 client.15849 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:03:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.155+0000 7f1680c00700 1 -- 172.21.15.16:0/283393773 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b42e6d1570 con 0x55b42e834b80 2024-05-27T07:03:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f1680c00700 10 client.15849 handle_command_reply: tid=0 2024-05-27T07:03:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 client.15849 shutdown 2024-05-27T07:03:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 msgr2=0x55b42e7d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f169000b0a0 tx=0x7f169000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 shutdown_connections 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55b42e834b80 0x55b42e836fa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b42e7c9ef0 0x55b42e7d1560 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b42e7b05c0 0x55b42e7c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 --2- 172.21.15.16:0/283393773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b42e7b64e0 0x55b42e7d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 >> 172.21.15.16:0/283393773 conn(0x55b42e688fc0 msgr2=0x55b42e7b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f1680c00700 0 client.15849 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 shutdown_connections 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.156+0000 7f16ac036b80 1 -- 172.21.15.16:0/283393773 wait complete. 2024-05-27T07:03:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.157+0000 7f16ac036b80 20 client.15849 trim_cache size 0 max 16384 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.157+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 msgr2=0x7f167c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.157+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 0x7f167c0b4fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1690009900 tx=0x7f1690006000 comp rx=0 tx=0).stop 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.157+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 msgr2=0x7f169819d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.157+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 0x7f169819d4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f168400e9c0 tx=0x7f168400ecd0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 shutdown_connections 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f167c0b2aa0 0x7f167c0b4fb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698108730 0x7f16981a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16981034b0 0x7f169819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 --2- 172.21.15.16:0/771714636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1698107de0 0x7f169819d4c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 >> 172.21.15.16:0/771714636 conn(0x7f16980fedb0 msgr2=0x7f1698109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 shutdown_connections 2024-05-27T07:03:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:27.158+0000 7f169d400700 1 -- 172.21.15.16:0/771714636 wait complete. 2024-05-27T07:03:27.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:27 smithi164 ceph-mon[35327]: pgmap v79: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-05-27T07:03:27.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:27 smithi016 ceph-mon[31415]: pgmap v79: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-05-27T07:03:27.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:27 smithi069 ceph-mon[35276]: pgmap v79: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-05-27T07:03:29.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:03:29.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:29 smithi016 ceph-mon[31415]: pgmap v80: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:29.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:29 smithi164 ceph-mon[35327]: pgmap v80: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:29.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:29 smithi069 ceph-mon[35276]: pgmap v80: 129 pgs: 129 active+clean; 2.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 82 MiB/s wr, 50 op/s 2024-05-27T07:03:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:31 smithi164 ceph-mon[35327]: pgmap v81: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 126 MiB/s wr, 76 op/s 2024-05-27T07:03:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:31 smithi016 ceph-mon[31415]: pgmap v81: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 126 MiB/s wr, 76 op/s 2024-05-27T07:03:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:31 smithi069 ceph-mon[35276]: pgmap v81: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 126 MiB/s wr, 76 op/s 2024-05-27T07:03:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 -- 172.21.15.16:0/4031389661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec108ef0 msgr2=0x7f80ec109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 --2- 172.21.15.16:0/4031389661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec108ef0 0x7f80ec109300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f80d80099c0 tx=0x7f80d8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 -- 172.21.15.16:0/4031389661 shutdown_connections 2024-05-27T07:03:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 --2- 172.21.15.16:0/4031389661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec109840 0x7f80ec1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 --2- 172.21.15.16:0/4031389661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec1045c0 0x7f80ec1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 --2- 172.21.15.16:0/4031389661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec108ef0 0x7f80ec109300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.946+0000 7f80f3c00700 1 -- 172.21.15.16:0/4031389661 >> 172.21.15.16:0/4031389661 conn(0x7f80ec0ffe90 msgr2=0x7f80ec1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.947+0000 7f80f3c00700 1 -- 172.21.15.16:0/4031389661 shutdown_connections 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.947+0000 7f80f3c00700 1 -- 172.21.15.16:0/4031389661 wait complete. 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.950+0000 7f80f3c00700 1 Processor -- start 2024-05-27T07:03:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.951+0000 7f80f3c00700 1 -- start start 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.951+0000 7f80f3c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 0x7f80ec076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f3c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec109840 0x7f80ec077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f3c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f3c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec114300 con 0x7f80ec077640 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f3c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec114180 con 0x7f80ec1045c0 2024-05-27T07:03:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f3c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec114480 con 0x7f80ec109840 2024-05-27T07:03:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f1c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f1c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44754/0 (socket says 172.21.15.16:44754) 2024-05-27T07:03:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80f1c00700 1 -- 172.21.15.16:0/916577828 learned_addr learned my addr 172.21.15.16:0/916577828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.952+0000 7f80ebe00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec109840 0x7f80ec077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.953+0000 7f80f1c00700 1 -- 172.21.15.16:0/916577828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec109840 msgr2=0x7f80ec077100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.953+0000 7f80f1c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec109840 0x7f80ec077100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.953+0000 7f80f1c00700 1 -- 172.21.15.16:0/916577828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 msgr2=0x7f80ec076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.954+0000 7f80f1200700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 0x7f80ec076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.954+0000 7f80f1c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 0x7f80ec076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.954+0000 7f80f1c00700 1 -- 172.21.15.16:0/916577828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec1b1de0 con 0x7f80ec077640 2024-05-27T07:03:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.955+0000 7f80f1200700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 0x7f80ec076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.955+0000 7f80f1c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f80e400d980 tx=0x7f80e400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.957+0000 7f80e9600700 1 -- 172.21.15.16:0/916577828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4017070 con 0x7f80ec077640 2024-05-27T07:03:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.957+0000 7f80e9600700 1 -- 172.21.15.16:0/916577828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e4009dd0 con 0x7f80ec077640 2024-05-27T07:03:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.957+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec1b20d0 con 0x7f80ec077640 2024-05-27T07:03:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.957+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec1b24d0 con 0x7f80ec077640 2024-05-27T07:03:32.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.958+0000 7f80e9600700 1 -- 172.21.15.16:0/916577828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4005260 con 0x7f80ec077640 2024-05-27T07:03:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.964+0000 7f80e9600700 1 -- 172.21.15.16:0/916577828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f80e40054e0 con 0x7f80ec077640 2024-05-27T07:03:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.965+0000 7f81026aab80 1 Processor -- start 2024-05-27T07:03:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.965+0000 7f81026aab80 1 -- start start 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 0x55daf2ae6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ae0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2ae09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55daf2a05e40 con 0x55daf2958e40 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55daf29f59e0 con 0x55daf2ae6540 2024-05-27T07:03:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.966+0000 7f81026aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55daf29f9a20 con 0x55daf2ae05c0 2024-05-27T07:03:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.967+0000 7f80f1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ae0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.967+0000 7f80ebe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 0x55daf2ae6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.967+0000 7f80e9600700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 0x7f80d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.967+0000 7f80f1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ae0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42612/0 (socket says 172.21.15.16:42612) 2024-05-27T07:03:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.968+0000 7f80f1200700 1 -- 172.21.15.16:0/987300046 learned_addr learned my addr 172.21.15.16:0/987300046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.968+0000 7f80ebe00700 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2ae09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.968+0000 7f80e9600700 1 -- 172.21.15.16:0/916577828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f80e40e28a0 con 0x7f80ec077640 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80f1c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 0x7f80d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80ebe00700 1 -- 172.21.15.16:0/987300046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 msgr2=0x55daf2ae09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80ebe00700 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2ae09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80ebe00700 1 -- 172.21.15.16:0/987300046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 msgr2=0x55daf2ae0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80ebe00700 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ae0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.969+0000 7f80ebe00700 1 -- 172.21.15.16:0/987300046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf29be420 con 0x55daf2958e40 2024-05-27T07:03:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.970+0000 7f80ebe00700 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 0x55daf2ae6000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f80ec1053f0 tx=0x7f80d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cbaf64c9a1c78a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.970+0000 7f80f1c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 0x7f80d40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80e40a5870 tx=0x7f80e40a5040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.970+0000 7f80de000700 1 -- 172.21.15.16:0/987300046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8005270 con 0x55daf2958e40 2024-05-27T07:03:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.971+0000 7f80de000700 1 -- 172.21.15.16:0/987300046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d800bc20 con 0x55daf2958e40 2024-05-27T07:03:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.971+0000 7f80de000700 1 -- 172.21.15.16:0/987300046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80038e0 con 0x55daf2958e40 2024-05-27T07:03:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 -- 172.21.15.16:0/987300046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 msgr2=0x55daf2ae6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 0x55daf2ae6000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f80ec1053f0 tx=0x7f80d8009580 comp rx=0 tx=0).stop 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 -- 172.21.15.16:0/987300046 shutdown_connections 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2ae09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2958e40 0x55daf2ae6000 secure :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f80ec1053f0 tx=0x7f80d8009580 comp rx=0 tx=0).stop 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 --2- 172.21.15.16:0/987300046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ae0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.972+0000 7f81026aab80 1 -- 172.21.15.16:0/987300046 >> 172.21.15.16:0/987300046 conn(0x55daf29b8fc0 msgr2=0x55daf2ad28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.973+0000 7f81026aab80 1 -- 172.21.15.16:0/987300046 shutdown_connections 2024-05-27T07:03:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.973+0000 7f81026aab80 1 -- 172.21.15.16:0/987300046 wait complete. 2024-05-27T07:03:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.973+0000 7f81026aab80 1 -- start start 2024-05-27T07:03:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.974+0000 7f81026aab80 1 Processor -- start 2024-05-27T07:03:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2aea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2aea810 0x55daf2adb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55daf28fbfa0 con 0x55daf2ae05c0 2024-05-27T07:03:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55daf2996760 con 0x55daf2ae6540 2024-05-27T07:03:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55daf2aecfb0 con 0x55daf2aea810 2024-05-27T07:03:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f81026aab80 10 client.0 _ll_register_callbacks cb 0x55daf299fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f80ebe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f80ebe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44770/0 (socket says 172.21.15.16:44770) 2024-05-27T07:03:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.975+0000 7f80ebe00700 1 -- 172.21.15.16:0/3924245475 learned_addr learned my addr 172.21.15.16:0/3924245475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2aea810 0x55daf2adb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80f1200700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2aea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 -- 172.21.15.16:0/3924245475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2aea810 msgr2=0x55daf2adb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2aea810 0x55daf2adb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 -- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 msgr2=0x55daf2aea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2aea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.976+0000 7f80ebe00700 1 -- 172.21.15.16:0/3924245475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf29458d0 con 0x55daf2ae05c0 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.977+0000 7f80f1200700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2aea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.977+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006010 tx=0x7f80d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7dff6f6a2297b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.977+0000 7f80d2a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.977+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003890 con 0x55daf2ae05c0 2024-05-27T07:03:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.977+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d800bc20 con 0x55daf2ae05c0 2024-05-27T07:03:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.978+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f80d8003a30 con 0x55daf2ae05c0 2024-05-27T07:03:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.978+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80d8003c50 con 0x55daf2ae05c0 2024-05-27T07:03:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.978+0000 7f81026aab80 10 client.15861 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.978+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55daf29c6060 con 0x55daf2ae05c0 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d800bc20 con 0x55daf2ae05c0 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f80d800f460 con 0x55daf2ae05c0 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 10 client.15861 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 10 client.15861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 20 client.15861 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b02ac0 0x55daf2b04ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 4 client.15861 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.979+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf2a07d90 con 0x55daf2b02ac0 2024-05-27T07:03:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.980+0000 7f80f1200700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b02ac0 0x55daf2b04ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:32.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.981+0000 7f80f1200700 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b02ac0 0x55daf2b04ee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d394ba55c45b4861 server_cookie=d491113b2dc3d83b in_seq=0 out_seq=0 2024-05-27T07:03:32.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.981+0000 7f80d2a00700 10 client.15861 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:03:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.983+0000 7f80d2a00700 1 -- 172.21.15.16:0/3924245475 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55daf2a07d90 con 0x55daf2b02ac0 2024-05-27T07:03:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.983+0000 7f80d2a00700 10 client.15861 handle_command_reply: tid=0 2024-05-27T07:03:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.983+0000 7f81026aab80 1 client.15861 shutdown 2024-05-27T07:03:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.983+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 msgr2=0x55daf2ae9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.983+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006010 tx=0x7f80d8000bc0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 shutdown_connections 2024-05-27T07:03:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b02ac0 0x55daf2b04ee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2aea810 0x55daf2adb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f80d2a00700 0 client.15861 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ae9d90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 --2- 172.21.15.16:0/3924245475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2aea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 >> 172.21.15.16:0/3924245475 conn(0x55daf29b8fc0 msgr2=0x55daf2ad1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 shutdown_connections 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 1 -- 172.21.15.16:0/3924245475 wait complete. 2024-05-27T07:03:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:32.984+0000 7f81026aab80 20 client.15861 trim_cache size 0 max 16384 2024-05-27T07:03:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.012+0000 7f81026aab80 1 Processor -- start 2024-05-27T07:03:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 -- start start 2024-05-27T07:03:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2aec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2adffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00ed20 con 0x55daf2aea810 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc010860 con 0x55daf2ae6540 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.013+0000 7f81026aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00f3f0 con 0x55daf2ae05c0 2024-05-27T07:03:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.014+0000 7f80ebe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.014+0000 7f80f1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2adffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.014+0000 7f80ebe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44782/0 (socket says 172.21.15.16:44782) 2024-05-27T07:03:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.014+0000 7f80ebe00700 1 -- 172.21.15.16:0/902138674 learned_addr learned my addr 172.21.15.16:0/902138674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.014+0000 7f80ebe00700 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2aec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 -- 172.21.15.16:0/902138674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 msgr2=0x55daf2aec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2aec230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 -- 172.21.15.16:0/902138674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 msgr2=0x55daf2adffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2adffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 -- 172.21.15.16:0/902138674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf2ae5970 con 0x55daf2aea810 2024-05-27T07:03:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.015+0000 7f80ebe00700 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f80d8035040 tx=0x7f80d8003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7879e7a634c8ed74 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.016+0000 7f80dd600700 1 -- 172.21.15.16:0/902138674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80051f0 con 0x55daf2aea810 2024-05-27T07:03:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.016+0000 7f80dd600700 1 -- 172.21.15.16:0/902138674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d800bc20 con 0x55daf2aea810 2024-05-27T07:03:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.016+0000 7f80dd600700 1 -- 172.21.15.16:0/902138674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d800fa70 con 0x55daf2aea810 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 -- 172.21.15.16:0/902138674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 msgr2=0x55daf2ae8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f80d8035040 tx=0x7f80d8003df0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 -- 172.21.15.16:0/902138674 shutdown_connections 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2aea810 0x55daf2ae8c80 secure :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f80d8035040 tx=0x7f80d8003df0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2ae05c0 0x55daf2aec230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 --2- 172.21.15.16:0/902138674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2adffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 -- 172.21.15.16:0/902138674 >> 172.21.15.16:0/902138674 conn(0x55daf29b8fc0 msgr2=0x55daf2ae8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 -- 172.21.15.16:0/902138674 shutdown_connections 2024-05-27T07:03:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.018+0000 7f81026aab80 1 -- 172.21.15.16:0/902138674 wait complete. 2024-05-27T07:03:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.019+0000 7f81026aab80 1 -- start start 2024-05-27T07:03:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 Processor -- start 2024-05-27T07:03:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ad8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2b65360 0x55daf2b65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc010190 con 0x55daf2ae05c0 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00fac0 con 0x55daf2ae6540 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00ccc0 con 0x55daf2b65360 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.020+0000 7f81026aab80 10 client.0 _ll_register_callbacks cb 0x55daf299fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.021+0000 7f80f1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.021+0000 7f80f1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42630/0 (socket says 172.21.15.16:42630) 2024-05-27T07:03:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.021+0000 7f80f1200700 1 -- 172.21.15.16:0/3552675479 learned_addr learned my addr 172.21.15.16:0/3552675479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:33.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ad8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80f1200700 1 -- 172.21.15.16:0/3552675479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2b65360 msgr2=0x55daf2b65770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:03:33.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80f1200700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2b65360 0x55daf2b65770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80f1200700 1 -- 172.21.15.16:0/3552675479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 msgr2=0x55daf2ad8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80f1200700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ad8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80f1200700 1 -- 172.21.15.16:0/3552675479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf2b65ce0 con 0x55daf2ae6540 2024-05-27T07:03:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.022+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ad8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.023+0000 7f80f1200700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f80 tx=0x7f80e0010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeeaf723cc66a39d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.023+0000 7f80d2a00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:03:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.024+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0007e60 con 0x55daf2ae6540 2024-05-27T07:03:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.024+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f80e0018c20 con 0x55daf2ae6540 2024-05-27T07:03:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.024+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d800bc20 con 0x55daf2ae6540 2024-05-27T07:03:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.025+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80e0018e60 con 0x55daf2ae6540 2024-05-27T07:03:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.025+0000 7f81026aab80 10 client.35597 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.025+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55daf2b6d050 con 0x55daf2ae6540 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.026+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d800bc20 con 0x55daf2ae6540 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.026+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f80e002a030 con 0x55daf2ae6540 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 10 client.35597 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 10 client.35597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 20 client.35597 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b6d7e0 0x55daf2b6fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 4 client.35597 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:03:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.027+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55daf2a01570 con 0x55daf2b6d7e0 2024-05-27T07:03:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.028+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b6d7e0 0x55daf2b6fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.030+0000 7f80ebe00700 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b6d7e0 0x55daf2b6fc00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bd238f69dec412f server_cookie=c48490f8a485b3b in_seq=0 out_seq=0 2024-05-27T07:03:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.030+0000 7f80d2a00700 10 client.35597 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:03:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.030+0000 7f80d2a00700 1 -- 172.21.15.16:0/3552675479 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55daf2a01570 con 0x55daf2b6d7e0 2024-05-27T07:03:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f80d2a00700 10 client.35597 handle_command_reply: tid=0 2024-05-27T07:03:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 client.35597 shutdown 2024-05-27T07:03:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 msgr2=0x55daf2ad91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f80 tx=0x7f80e0010ce0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 shutdown_connections 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55daf2b6d7e0 0x55daf2b6fc00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55daf2b65360 0x55daf2b65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55daf2ae05c0 0x55daf2ad8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 --2- 172.21.15.16:0/3552675479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55daf2ae6540 0x55daf2ad91f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 >> 172.21.15.16:0/3552675479 conn(0x55daf29b8fc0 msgr2=0x55daf2ae8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f80d2a00700 0 client.35597 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:03:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.031+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 shutdown_connections 2024-05-27T07:03:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.032+0000 7f81026aab80 1 -- 172.21.15.16:0/3552675479 wait complete. 2024-05-27T07:03:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.032+0000 7f81026aab80 20 client.35597 trim_cache size 0 max 16384 2024-05-27T07:03:33.025 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 msgr2=0x7f80d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 0x7f80d40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80e40a5870 tx=0x7f80e40a5040 comp rx=0 tx=0).stop 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 msgr2=0x7f80ec1b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f80e400d980 tx=0x7f80e400dd40 comp rx=0 tx=0).stop 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 shutdown_connections 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f80d40b29f0 0x7f80d40b4f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec077640 0x7f80ec1b1820 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80ec1045c0 0x7f80ec076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 --2- 172.21.15.16:0/916577828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80ec109840 0x7f80ec077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.033+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 >> 172.21.15.16:0/916577828 conn(0x7f80ec0ffe90 msgr2=0x7f80ec1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.034+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 shutdown_connections 2024-05-27T07:03:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:33.034+0000 7f80f3c00700 1 -- 172.21.15.16:0/916577828 wait complete. 2024-05-27T07:03:33.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:33 smithi164 ceph-mon[35327]: pgmap v82: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:33.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:33 smithi016 ceph-mon[31415]: pgmap v82: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:33.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:33 smithi069 ceph-mon[35276]: pgmap v82: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:34.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:03:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:35.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:35 smithi164 ceph-mon[35327]: pgmap v83: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:35.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:35 smithi016 ceph-mon[31415]: pgmap v83: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:35.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:35 smithi069 ceph-mon[35276]: pgmap v83: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:37.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:37 smithi016 ceph-mon[31415]: pgmap v84: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.839+0000 7ff6c2400700 1 -- 172.21.15.16:0/3459224285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc106eb0 msgr2=0x7ff6bc113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.839+0000 7ff6c2400700 1 --2- 172.21.15.16:0/3459224285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc113340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400b0a0 tx=0x7ff6b400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 -- 172.21.15.16:0/3459224285 shutdown_connections 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 --2- 172.21.15.16:0/3459224285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc113340 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 --2- 172.21.15.16:0/3459224285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 --2- 172.21.15.16:0/3459224285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 0x7ff6bc106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 -- 172.21.15.16:0/3459224285 >> 172.21.15.16:0/3459224285 conn(0x7ff6bc102040 msgr2=0x7ff6bc104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 -- 172.21.15.16:0/3459224285 shutdown_connections 2024-05-27T07:03:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.840+0000 7ff6c2400700 1 -- 172.21.15.16:0/3459224285 wait complete. 2024-05-27T07:03:37.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:37 smithi164 ceph-mon[35327]: pgmap v84: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.842+0000 7ff6c2400700 1 Processor -- start 2024-05-27T07:03:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.842+0000 7ff6c2400700 1 -- start start 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 0x7ff6bc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1165d0 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc116450 con 0x7ff6bc106eb0 2024-05-27T07:03:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.843+0000 7ff6c2400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc116750 con 0x7ff6bc106560 2024-05-27T07:03:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.844+0000 7ff6bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.844+0000 7ff6bbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44818/0 (socket says 172.21.15.16:44818) 2024-05-27T07:03:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.844+0000 7ff6bbe00700 1 -- 172.21.15.16:0/216144028 learned_addr learned my addr 172.21.15.16:0/216144028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.844+0000 7ff6bb400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 0x7ff6bc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6baa00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6bbe00700 1 -- 172.21.15.16:0/216144028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 msgr2=0x7ff6bc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6bbe00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 0x7ff6bc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6bbe00700 1 -- 172.21.15.16:0/216144028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc106eb0 msgr2=0x7ff6bc19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6bbe00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.845+0000 7ff6bbe00700 1 -- 172.21.15.16:0/216144028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1a1db0 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.846+0000 7ff6bbe00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400b070 tx=0x7ff6b4004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.847+0000 7ff6b3e00700 1 -- 172.21.15.16:0/216144028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4021070 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.847+0000 7ff6b3e00700 1 -- 172.21.15.16:0/216144028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b4007c00 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.847+0000 7ff6b3e00700 1 -- 172.21.15.16:0/216144028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004780 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.847+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1a2040 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.847+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc1a24d0 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.853+0000 7ff6b3e00700 1 -- 172.21.15.16:0/216144028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff6b401d450 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.853+0000 7ff6d0ff7b80 1 Processor -- start 2024-05-27T07:03:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 -- start start 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d608092870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d607fc0e40 con 0x55d608098d30 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d607fb09e0 con 0x55d608092db0 2024-05-27T07:03:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.854+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d607fb4a20 con 0x55d607f13e40 2024-05-27T07:03:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d608092870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47680/0 (socket says 172.21.15.16:47680) 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6baa00700 1 -- 172.21.15.16:0/3820310605 learned_addr learned my addr 172.21.15.16:0/3820310605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.855+0000 7ff6b3e00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 0x7ff6a00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.856+0000 7ff6b3e00700 1 -- 172.21.15.16:0/216144028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff6b400f020 con 0x7ff6bc10c5a0 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.856+0000 7ff6baa00700 1 -- 172.21.15.16:0/3820310605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 msgr2=0x55d6080931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.856+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.856+0000 7ff6baa00700 1 -- 172.21.15.16:0/3820310605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 msgr2=0x55d608092870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.857+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d608092870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.857+0000 7ff6baa00700 1 -- 172.21.15.16:0/3820310605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d607f79420 con 0x55d607f13e40 2024-05-27T07:03:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.857+0000 7ff6bb400700 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d608092870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.857+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014990 tx=0x7ff6ac00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72fb39a94b2225af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.858+0000 7ff6b0c00700 1 -- 172.21.15.16:0/3820310605 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004830 con 0x55d607f13e40 2024-05-27T07:03:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.858+0000 7ff6b0c00700 1 -- 172.21.15.16:0/3820310605 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ac005020 con 0x55d607f13e40 2024-05-27T07:03:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.858+0000 7ff6b0c00700 1 -- 172.21.15.16:0/3820310605 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004e60 con 0x55d607f13e40 2024-05-27T07:03:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.858+0000 7ff6bbe00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 0x7ff6a00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.859+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3820310605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 msgr2=0x55d6080987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.859+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014990 tx=0x7ff6ac00c490 comp rx=0 tx=0).stop 2024-05-27T07:03:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6bbe00700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 0x7ff6a00b5050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40aac20 tx=0x7ff6b4009b80 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3820310605 shutdown_connections 2024-05-27T07:03:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d6080987f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3820310605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d608092870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3820310605 >> 172.21.15.16:0/3820310605 conn(0x55d607f73fc0 msgr2=0x55d60808d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3820310605 shutdown_connections 2024-05-27T07:03:37.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.860+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3820310605 wait complete. 2024-05-27T07:03:37.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.861+0000 7ff6d0ff7b80 1 -- start start 2024-05-27T07:03:37.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.861+0000 7ff6d0ff7b80 1 Processor -- start 2024-05-27T07:03:37.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d60809c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d60809c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d607eb6fa0 con 0x55d608098d30 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d607f51760 con 0x55d608092db0 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d60809f800 con 0x55d607f13e40 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6d0ff7b80 10 client.0 _ll_register_callbacks cb 0x55d607f5ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.862+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.863+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44842/0 (socket says 172.21.15.16:44842) 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.863+0000 7ff6baa00700 1 -- 172.21.15.16:0/1620470668 learned_addr learned my addr 172.21.15.16:0/1620470668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.863+0000 7ff6bb400700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d60809c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d60809c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 -- 172.21.15.16:0/1620470668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 msgr2=0x55d60809c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d60809c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 -- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 msgr2=0x55d60809c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d60809c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.864+0000 7ff6baa00700 1 -- 172.21.15.16:0/1620470668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d607f008d0 con 0x55d608098d30 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.865+0000 7ff6baa00700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014440 tx=0x7ff6ac005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c696a3457d1b1ec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.865+0000 7ff6a9600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.865+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004b60 con 0x55d608098d30 2024-05-27T07:03:37.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.865+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff6ac004d00 con 0x55d608098d30 2024-05-27T07:03:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.865+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac005020 con 0x55d608098d30 2024-05-27T07:03:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac011500 con 0x55d608098d30 2024-05-27T07:03:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6d0ff7b80 10 client.15876 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d607f81060 con 0x55d608098d30 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ac011b30 con 0x55d608098d30 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff6ac0127f0 con 0x55d608098d30 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6d0ff7b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.866+0000 7ff6d0ff7b80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.867+0000 7ff6d0ff7b80 20 client.15876 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.867+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6080b4d90 0x55d6080b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.867+0000 7ff6d0ff7b80 4 client.15876 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:37.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.867+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff6ac005020 con 0x55d6080b4d90 2024-05-27T07:03:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.867+0000 7ff6bb400700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6080b4d90 0x55d6080b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.869+0000 7ff6a9600700 10 client.15876 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:37.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.869+0000 7ff6bb400700 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6080b4d90 0x55d6080b71b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c133231e03c3d756 server_cookie=e05f4c901fbd89e9 in_seq=0 out_seq=0 2024-05-27T07:03:37.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.872+0000 7ff6a9600700 1 -- 172.21.15.16:0/1620470668 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7ff6ac005020 con 0x55d6080b4d90 2024-05-27T07:03:37.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.872+0000 7ff6a9600700 10 client.15876 handle_command_reply: tid=0 2024-05-27T07:03:37.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.872+0000 7ff6d0ff7b80 1 client.15876 shutdown 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.872+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 msgr2=0x55d60809cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.872+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014440 tx=0x7ff6ac005f60 comp rx=0 tx=0).stop 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 shutdown_connections 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6080b4d90 0x55d6080b71b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d608098d30 0x55d60809cf30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d607f13e40 0x55d60809c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/1620470668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d60809c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 >> 172.21.15.16:0/1620470668 conn(0x55d607f73fc0 msgr2=0x55d60808cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6a9600700 0 client.15876 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 shutdown_connections 2024-05-27T07:03:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/1620470668 wait complete. 2024-05-27T07:03:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.873+0000 7ff6d0ff7b80 20 client.15876 trim_cache size 0 max 16384 2024-05-27T07:03:37.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:37 smithi069 ceph-mon[35276]: pgmap v84: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.900+0000 7ff6d0ff7b80 1 Processor -- start 2024-05-27T07:03:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.900+0000 7ff6d0ff7b80 1 -- start start 2024-05-27T07:03:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608092db0 0x55d608096c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608098d30 0x55d6080a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1ae200 con 0x55d607f13e40 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1afd40 con 0x55d608098d30 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.901+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1ae8d0 con 0x55d608092db0 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.902+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.902+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44858/0 (socket says 172.21.15.16:44858) 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.902+0000 7ff6baa00700 1 -- 172.21.15.16:0/3057402769 learned_addr learned my addr 172.21.15.16:0/3057402769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.902+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608098d30 0x55d6080a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.902+0000 7ff6bb400700 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608092db0 0x55d608096c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 -- 172.21.15.16:0/3057402769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608092db0 msgr2=0x55d608096c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608092db0 0x55d608096c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 -- 172.21.15.16:0/3057402769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608098d30 msgr2=0x55d6080a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608098d30 0x55d6080a8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 -- 172.21.15.16:0/3057402769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d60809aa20 con 0x55d607f13e40 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.903+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014990 tx=0x7ff6ac015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3174e9609f388d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.904+0000 7ff6abe00700 1 -- 172.21.15.16:0/3057402769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac032080 con 0x55d607f13e40 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.904+0000 7ff6abe00700 1 -- 172.21.15.16:0/3057402769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ac004030 con 0x55d607f13e40 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.904+0000 7ff6abe00700 1 -- 172.21.15.16:0/3057402769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac011500 con 0x55d607f13e40 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3057402769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 msgr2=0x55d608092870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac014990 tx=0x7ff6ac015f90 comp rx=0 tx=0).stop 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3057402769 shutdown_connections 2024-05-27T07:03:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608098d30 0x55d6080a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d608092870 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3057402769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608092db0 0x55d608096c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.905+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3057402769 >> 172.21.15.16:0/3057402769 conn(0x55d607f73fc0 msgr2=0x55d60809c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.906+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3057402769 shutdown_connections 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.906+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3057402769 wait complete. 2024-05-27T07:03:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.906+0000 7ff6d0ff7b80 1 -- start start 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 Processor -- start 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d6080ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608098d30 0x55d6080b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1af670 con 0x55d607f13e40 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.907+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1aefa0 con 0x55d608092db0 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6d0ff7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1ac160 con 0x55d608098d30 2024-05-27T07:03:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6d0ff7b80 10 client.0 _ll_register_callbacks cb 0x55d608093bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6bb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608098d30 0x55d6080b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d6080ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6baa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d6080ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44862/0 (socket says 172.21.15.16:44862) 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.908+0000 7ff6baa00700 1 -- 172.21.15.16:0/3836743574 learned_addr learned my addr 172.21.15.16:0/3836743574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.909+0000 7ff6bb400700 1 -- 172.21.15.16:0/3836743574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608098d30 msgr2=0x55d6080b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.909+0000 7ff6bb400700 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608098d30 0x55d6080b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.909+0000 7ff6bb400700 1 -- 172.21.15.16:0/3836743574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 msgr2=0x55d6080ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.909+0000 7ff6bb400700 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d6080ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.909+0000 7ff6bb400700 1 -- 172.21.15.16:0/3836743574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6080b72e0 con 0x55d608092db0 2024-05-27T07:03:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.910+0000 7ff6bb400700 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080b6da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55d607eb7030 tx=0x7ff6a400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cfec9a422d68afb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.910+0000 7ff6a9600700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:03:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.911+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4003dd0 con 0x55d608092db0 2024-05-27T07:03:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.911+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff6a401d420 con 0x55d608092db0 2024-05-27T07:03:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.912+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac004030 con 0x55d608092db0 2024-05-27T07:03:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.912+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6a401d620 con 0x55d608092db0 2024-05-27T07:03:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.912+0000 7ff6d0ff7b80 10 client.35615 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.912+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d608126c90 con 0x55d608092db0 2024-05-27T07:03:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ac004030 con 0x55d608092db0 2024-05-27T07:03:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff6a4011a20 con 0x55d608092db0 2024-05-27T07:03:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 10 client.35615 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 10 client.35615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 20 client.35615 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6081273a0 0x55d6081297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 4 client.35615 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.913+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d607fc2d90 con 0x55d6081273a0 2024-05-27T07:03:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.914+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6081273a0 0x55d6081297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.915+0000 7ff6a9600700 10 client.35615 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.915+0000 7ff6baa00700 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6081273a0 0x55d6081297c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f87c9d0398a115 server_cookie=30a44d65a31a0508 in_seq=0 out_seq=0 2024-05-27T07:03:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6a9600700 1 -- 172.21.15.16:0/3836743574 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d607fc2d90 con 0x55d6081273a0 2024-05-27T07:03:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6a9600700 10 client.35615 handle_command_reply: tid=0 2024-05-27T07:03:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 client.35615 shutdown 2024-05-27T07:03:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 msgr2=0x55d6080b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080b6da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55d607eb7030 tx=0x7ff6a400dc00 comp rx=0 tx=0).stop 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 shutdown_connections 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d6081273a0 0x55d6081297c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d608098d30 0x55d6080b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d607f13e40 0x55d6080ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 --2- 172.21.15.16:0/3836743574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d608092db0 0x55d6080b6da0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.916+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 >> 172.21.15.16:0/3836743574 conn(0x55d607f73fc0 msgr2=0x55d6080969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.917+0000 7ff6a9600700 0 client.35615 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.917+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 shutdown_connections 2024-05-27T07:03:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.917+0000 7ff6d0ff7b80 1 -- 172.21.15.16:0/3836743574 wait complete. 2024-05-27T07:03:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.917+0000 7ff6d0ff7b80 20 client.35615 trim_cache size 0 max 16384 2024-05-27T07:03:37.909 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 msgr2=0x7ff6a00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 0x7ff6a00b5050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40aac20 tx=0x7ff6b4009b80 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 msgr2=0x7ff6bc1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400b070 tx=0x7ff6b4004e00 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 shutdown_connections 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff6a00b2b40 0x7ff6a00b5050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc10c5a0 0x7ff6bc1a1710 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6bc106560 0x7ff6bc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 --2- 172.21.15.16:0/216144028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6bc106eb0 0x7ff6bc19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.918+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 >> 172.21.15.16:0/216144028 conn(0x7ff6bc102040 msgr2=0x7ff6bc1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.919+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 shutdown_connections 2024-05-27T07:03:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:37.919+0000 7ff6c2400700 1 -- 172.21.15.16:0/216144028 wait complete. 2024-05-27T07:03:39.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:39 smithi164 ceph-mon[35327]: pgmap v85: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:39.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:39 smithi016 ceph-mon[31415]: pgmap v85: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:39.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:39 smithi069 ceph-mon[35276]: pgmap v85: 129 pgs: 129 active+clean; 3.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:40.079 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:03:40.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:03:40.689 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:41.533 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:41 smithi016 ceph-mon[31415]: pgmap v86: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:41.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:41 smithi164 ceph-mon[35327]: pgmap v86: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:41.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:41 smithi069 ceph-mon[35276]: pgmap v86: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 127 MiB/s wr, 76 op/s 2024-05-27T07:03:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 -- 172.21.15.16:0/1586941589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 msgr2=0x7fecdc110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 --2- 172.21.15.16:0/1586941589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc110bd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feccc00b0a0 tx=0x7feccc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 -- 172.21.15.16:0/1586941589 shutdown_connections 2024-05-27T07:03:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 --2- 172.21.15.16:0/1586941589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc110bd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 --2- 172.21.15.16:0/1586941589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105280 0x7fecdc105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 --2- 172.21.15.16:0/1586941589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105bd0 0x7fecdc10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.299+0000 7fecdb200700 1 -- 172.21.15.16:0/1586941589 >> 172.21.15.16:0/1586941589 conn(0x7fecdc078ab0 msgr2=0x7fecdc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.300+0000 7fecdb200700 1 -- 172.21.15.16:0/1586941589 shutdown_connections 2024-05-27T07:03:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.300+0000 7fecdb200700 1 -- 172.21.15.16:0/1586941589 wait complete. 2024-05-27T07:03:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.301+0000 7fecdb200700 1 Processor -- start 2024-05-27T07:03:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 -- start start 2024-05-27T07:03:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 0x7fecdc1a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105bd0 0x7fecdc1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1034b0 con 0x7fecdc10e6f0 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc103330 con 0x7fecdc105280 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.302+0000 7fecdb200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc103630 con 0x7fecdc105bd0 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.303+0000 7fecda800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.303+0000 7fecda800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52058/0 (socket says 172.21.15.16:52058) 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.303+0000 7fecda800700 1 -- 172.21.15.16:0/2421096835 learned_addr learned my addr 172.21.15.16:0/2421096835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.303+0000 7fecd9e00700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 0x7fecdc1a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.303+0000 7fecd9400700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105bd0 0x7fecdc1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecda800700 1 -- 172.21.15.16:0/2421096835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105bd0 msgr2=0x7fecdc1a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecda800700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105bd0 0x7fecdc1a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecda800700 1 -- 172.21.15.16:0/2421096835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 msgr2=0x7fecdc1a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecda800700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 0x7fecdc1a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecda800700 1 -- 172.21.15.16:0/2421096835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc19d3a0 con 0x7fecdc10e6f0 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.304+0000 7fecd9e00700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 0x7fecdc1a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecda800700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feccc00b780 tx=0x7feccc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecd2a00700 1 -- 172.21.15.16:0/2421096835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc021070 con 0x7fecdc10e6f0 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecd2a00700 1 -- 172.21.15.16:0/2421096835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feccc007b80 con 0x7fecdc10e6f0 2024-05-27T07:03:43.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc19d630 con 0x7fecdc10e6f0 2024-05-27T07:03:43.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc1a9710 con 0x7fecdc10e6f0 2024-05-27T07:03:43.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.305+0000 7fecd2a00700 1 -- 172.21.15.16:0/2421096835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc005040 con 0x7fecdc10e6f0 2024-05-27T07:03:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.311+0000 7fecd2a00700 1 -- 172.21.15.16:0/2421096835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7feccc024070 con 0x7fecdc10e6f0 2024-05-27T07:03:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.313+0000 7fecefcbfb80 1 Processor -- start 2024-05-27T07:03:43.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.314+0000 7fecd2a00700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 0x7fecc00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.315+0000 7fecd2a00700 1 -- 172.21.15.16:0/2421096835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7feccc00f020 con 0x7fecdc10e6f0 2024-05-27T07:03:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.315+0000 7fecd9400700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 0x7fecc00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 -- start start 2024-05-27T07:03:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecd9400700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 0x7fecc00b5050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fecdc1104e0 tx=0x7fecc8009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x56515939d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651592b8ec0 con 0x56515920bec0 2024-05-27T07:03:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651592a8a60 con 0x56515938a300 2024-05-27T07:03:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.316+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651592acaa0 con 0x56515938a710 2024-05-27T07:03:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.317+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.317+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52064/0 (socket says 172.21.15.16:52064) 2024-05-27T07:03:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.317+0000 7fecd9e00700 1 -- 172.21.15.16:0/1238803832 learned_addr learned my addr 172.21.15.16:0/1238803832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.317+0000 7fecd9400700 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x56515939d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.317+0000 7fecda800700 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 -- 172.21.15.16:0/1238803832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 msgr2=0x56515939f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 -- 172.21.15.16:0/1238803832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 msgr2=0x56515939d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x56515939d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 -- 172.21.15.16:0/1238803832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651592714a0 con 0x56515920bec0 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.318+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fecd40115f0 tx=0x7fecd4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c27264d2c0d766 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.319+0000 7fecc7400700 1 -- 172.21.15.16:0/1238803832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4004230 con 0x56515920bec0 2024-05-27T07:03:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.319+0000 7fecc7400700 1 -- 172.21.15.16:0/1238803832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecd40043d0 con 0x56515920bec0 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.320+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1238803832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 msgr2=0x565159389dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.320+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fecd40115f0 tx=0x7fecd4011900 comp rx=0 tx=0).stop 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.320+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1238803832 shutdown_connections 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.320+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.320+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515920bec0 0x565159389dc0 secure :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fecd40115f0 tx=0x7fecd4011900 comp rx=0 tx=0).stop 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.321+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1238803832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x56515939d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.321+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1238803832 >> 172.21.15.16:0/1238803832 conn(0x56515926c040 msgr2=0x565159385db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.321+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1238803832 shutdown_connections 2024-05-27T07:03:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.321+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1238803832 wait complete. 2024-05-27T07:03:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.322+0000 7fecefcbfb80 1 -- start start 2024-05-27T07:03:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.322+0000 7fecefcbfb80 1 Processor -- start 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x5651593a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651591af020 con 0x56515938a300 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651592497e0 con 0x56515939d8d0 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.323+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651593a02d0 con 0x56515938a710 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.324+0000 7fecefcbfb80 10 client.0 _ll_register_callbacks cb 0x565159252cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.324+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.324+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52066/0 (socket says 172.21.15.16:52066) 2024-05-27T07:03:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.324+0000 7fecd9e00700 1 -- 172.21.15.16:0/222460347 learned_addr learned my addr 172.21.15.16:0/222460347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecda800700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x5651593a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 -- 172.21.15.16:0/222460347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 msgr2=0x56515939d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 -- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 msgr2=0x5651593a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x5651593a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 -- 172.21.15.16:0/222460347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651591f8950 con 0x56515938a300 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecda800700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x5651593a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecd9e00700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fecd401b040 tx=0x7fecd4005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21efdad98c53357b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.325+0000 7fecbbe00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.326+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4027070 con 0x56515938a300 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.326+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd40043d0 con 0x56515938a300 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.326+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fecd4004780 con 0x56515938a300 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.327+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fecd4033050 con 0x56515938a300 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.327+0000 7fecefcbfb80 10 client.15894 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.327+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651592790e0 con 0x56515938a300 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.327+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecd40043d0 con 0x56515938a300 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fecd4023920 con 0x56515938a300 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 20 client.15894 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5651593be690 0x5651593c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 4 client.15894 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.328+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651592bae10 con 0x5651593be690 2024-05-27T07:03:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.329+0000 7fecd9400700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5651593be690 0x5651593c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.330+0000 7fecd9400700 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5651593be690 0x5651593c0ab0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f5300bc676c117 server_cookie=884d00814d867fa4 in_seq=0 out_seq=0 2024-05-27T07:03:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.330+0000 7fecbbe00700 10 client.15894 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.332+0000 7fecbbe00700 1 -- 172.21.15.16:0/222460347 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5651592bae10 con 0x5651593be690 2024-05-27T07:03:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.332+0000 7fecbbe00700 10 client.15894 handle_command_reply: tid=0 2024-05-27T07:03:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.332+0000 7fecefcbfb80 1 client.15894 shutdown 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 msgr2=0x56515939ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fecd401b040 tx=0x7fecd4005320 comp rx=0 tx=0).stop 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 shutdown_connections 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5651593be690 0x5651593c0ab0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x5651593a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x56515939ce50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/222460347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 >> 172.21.15.16:0/222460347 conn(0x56515926c040 msgr2=0x565159385070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecbbe00700 0 client.15894 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.333+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 shutdown_connections 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.334+0000 7fecefcbfb80 1 -- 172.21.15.16:0/222460347 wait complete. 2024-05-27T07:03:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.334+0000 7fecefcbfb80 20 client.15894 trim_cache size 0 max 16384 2024-05-27T07:03:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.370+0000 7fecefcbfb80 1 Processor -- start 2024-05-27T07:03:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 -- start start 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x56515939f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00ed70 con 0x56515938a300 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0108b0 con 0x56515939d8d0 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.371+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00f440 con 0x56515938a710 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.372+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.372+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52068/0 (socket says 172.21.15.16:52068) 2024-05-27T07:03:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.372+0000 7fecd9e00700 1 -- 172.21.15.16:0/1367779496 learned_addr learned my addr 172.21.15.16:0/1367779496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.372+0000 7fecda800700 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x56515939f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.372+0000 7fecd9400700 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.373+0000 7fecd9e00700 1 -- 172.21.15.16:0/1367779496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 msgr2=0x56515939bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.373+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.373+0000 7fecd9e00700 1 -- 172.21.15.16:0/1367779496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 msgr2=0x56515939f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.373+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x56515939f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.373+0000 7fecd9e00700 1 -- 172.21.15.16:0/1367779496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515939fdc0 con 0x56515938a300 2024-05-27T07:03:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.374+0000 7fecd9e00700 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000c00 tx=0x7fecd4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24e54cce00cd278 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.374+0000 7fecc6a00700 1 -- 172.21.15.16:0/1367779496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4030020 con 0x56515938a300 2024-05-27T07:03:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.375+0000 7fecc6a00700 1 -- 172.21.15.16:0/1367779496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecd40238f0 con 0x56515938a300 2024-05-27T07:03:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.375+0000 7fecc6a00700 1 -- 172.21.15.16:0/1367779496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4030020 con 0x56515938a300 2024-05-27T07:03:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.376+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1367779496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 msgr2=0x565159385ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.376+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000c00 tx=0x7fecd4011900 comp rx=0 tx=0).stop 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1367779496 shutdown_connections 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515939d8d0 0x56515939f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a300 0x565159385ee0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/1367779496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515938a710 0x56515939bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1367779496 >> 172.21.15.16:0/1367779496 conn(0x56515926c040 msgr2=0x5651593857d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.377+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1367779496 shutdown_connections 2024-05-27T07:03:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.378+0000 7fecefcbfb80 1 -- 172.21.15.16:0/1367779496 wait complete. 2024-05-27T07:03:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.378+0000 7fecefcbfb80 1 -- start start 2024-05-27T07:03:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.379+0000 7fecefcbfb80 1 Processor -- start 2024-05-27T07:03:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.380+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.380+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a710 0x5651593a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecefcbfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515939d8d0 0x5651593a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecefcbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0101e0 con 0x56515938a710 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecd9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34098/0 (socket says 172.21.15.16:34098) 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecd9e00700 1 -- 172.21.15.16:0/2696443870 learned_addr learned my addr 172.21.15.16:0/2696443870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecd9400700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a710 0x5651593a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00fb10 con 0x56515938a300 2024-05-27T07:03:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00cd10 con 0x56515939d8d0 2024-05-27T07:03:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecefcbfb80 10 client.0 _ll_register_callbacks cb 0x565159385700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.381+0000 7fecda800700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515939d8d0 0x5651593a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.382+0000 7fecd9e00700 1 -- 172.21.15.16:0/2696443870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515939d8d0 msgr2=0x5651593a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.382+0000 7fecd9e00700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515939d8d0 0x5651593a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.382+0000 7fecd9e00700 1 -- 172.21.15.16:0/2696443870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a710 msgr2=0x5651593a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.382+0000 7fecd9e00700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a710 0x5651593a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.382+0000 7fecd9e00700 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651593a1ba0 con 0x56515938a300 2024-05-27T07:03:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.383+0000 7fecd9e00700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005590 tx=0x7fecd40093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25527be77f3ff100 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.384+0000 7fecbbe00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:03:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.384+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4030030 con 0x56515938a300 2024-05-27T07:03:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.384+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fecd4023460 con 0x56515938a300 2024-05-27T07:03:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.384+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd40238f0 con 0x56515938a300 2024-05-27T07:03:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.385+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fecd4027040 con 0x56515938a300 2024-05-27T07:03:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.385+0000 7fecefcbfb80 10 client.35630 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.385+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565159428f20 con 0x56515938a300 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecd40238f0 con 0x56515938a300 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fecd4003680 con 0x56515938a300 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 10 client.35630 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 10 client.35630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 20 client.35630 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565159429550 0x56515942b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 4 client.35630 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:03:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.386+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651592b45f0 con 0x565159429550 2024-05-27T07:03:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.388+0000 7fecd9400700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565159429550 0x56515942b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.389+0000 7fecd9400700 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565159429550 0x56515942b970 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25f14a75710c523 server_cookie=3b534cd84667c03c in_seq=0 out_seq=0 2024-05-27T07:03:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.389+0000 7fecbbe00700 10 client.35630 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.392+0000 7fecbbe00700 1 -- 172.21.15.16:0/2696443870 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651592b45f0 con 0x565159429550 2024-05-27T07:03:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.392+0000 7fecbbe00700 10 client.35630 handle_command_reply: tid=0 2024-05-27T07:03:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.392+0000 7fecefcbfb80 1 client.35630 shutdown 2024-05-27T07:03:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 msgr2=0x5651593a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005590 tx=0x7fecd40093c0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 shutdown_connections 2024-05-27T07:03:43.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565159429550 0x56515942b970 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56515939d8d0 0x5651593a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56515938a300 0x5651593a5c50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 --2- 172.21.15.16:0/2696443870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515938a710 0x5651593a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 >> 172.21.15.16:0/2696443870 conn(0x56515926c040 msgr2=0x56515938b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.393+0000 7fecbbe00700 0 client.35630 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:43.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.394+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 shutdown_connections 2024-05-27T07:03:43.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.394+0000 7fecefcbfb80 1 -- 172.21.15.16:0/2696443870 wait complete. 2024-05-27T07:03:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.396+0000 7fecefcbfb80 20 client.35630 trim_cache size 0 max 16384 2024-05-27T07:03:43.387 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:03:43.387 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:03:43.387 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e61461ca-e31d-436c-8afd-bd9332439e3d", 2024-05-27T07:03:43.387 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:03:43.387 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 msgr2=0x7fecc00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 0x7fecc00b5050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fecdc1104e0 tx=0x7fecc8009260 comp rx=0 tx=0).stop 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 msgr2=0x7fecdc19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feccc00b780 tx=0x7feccc004d00 comp rx=0 tx=0).stop 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 shutdown_connections 2024-05-27T07:03:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fecc00b2b40 0x7fecc00b5050 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc10e6f0 0x7fecdc19cc10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecdc105280 0x7fecdc1a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 --2- 172.21.15.16:0/2421096835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc105bd0 0x7fecdc1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.397+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 >> 172.21.15.16:0/2421096835 conn(0x7fecdc078ab0 msgr2=0x7fecdc101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.398+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 shutdown_connections 2024-05-27T07:03:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:43.398+0000 7fecdb200700 1 -- 172.21.15.16:0/2421096835 wait complete. 2024-05-27T07:03:43.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:43 smithi016 ceph-mon[31415]: pgmap v87: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:43.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:43 smithi164 ceph-mon[35327]: pgmap v87: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:43.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:43 smithi069 ceph-mon[35276]: pgmap v87: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:44.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:03:44.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:44 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:03:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:44 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:44 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:03:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:44 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:44 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:03:44.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:44 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:03:44.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:44 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:44 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:03:44.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:44 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:44 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:03:44.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:44 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:03:44.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:44 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:44 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:03:44.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:44 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:03:44.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:44 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:03:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:45.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:45 smithi016 ceph-mon[31415]: pgmap v88: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:45.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:45 smithi164 ceph-mon[35327]: pgmap v88: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:45.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:45 smithi069 ceph-mon[35276]: pgmap v88: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 84 MiB/s wr, 50 op/s 2024-05-27T07:03:47.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:47 smithi016 ceph-mon[31415]: pgmap v89: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 127 MiB/s wr, 77 op/s 2024-05-27T07:03:47.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:47 smithi164 ceph-mon[35327]: pgmap v89: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 127 MiB/s wr, 77 op/s 2024-05-27T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:47 smithi069 ceph-mon[35276]: pgmap v89: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 127 MiB/s wr, 77 op/s 2024-05-27T07:03:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.045+0000 7fb28dc00700 1 -- 172.21.15.16:0/292570505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288108690 msgr2=0x7fb288108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.045+0000 7fb28dc00700 1 --2- 172.21.15.16:0/292570505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288108690 0x7fb288108b60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b0a0 tx=0x7fb28000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.046+0000 7fb28dc00700 1 -- 172.21.15.16:0/292570505 shutdown_connections 2024-05-27T07:03:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.046+0000 7fb28dc00700 1 --2- 172.21.15.16:0/292570505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288108690 0x7fb288108b60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.046+0000 7fb28dc00700 1 --2- 172.21.15.16:0/292570505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103240 0x7fb288103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.046+0000 7fb28dc00700 1 --2- 172.21.15.16:0/292570505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288103b90 0x7fb2881080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.046+0000 7fb28dc00700 1 -- 172.21.15.16:0/292570505 >> 172.21.15.16:0/292570505 conn(0x7fb2880fedb0 msgr2=0x7fb2881011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.047+0000 7fb28dc00700 1 -- 172.21.15.16:0/292570505 shutdown_connections 2024-05-27T07:03:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.047+0000 7fb28dc00700 1 -- 172.21.15.16:0/292570505 wait complete. 2024-05-27T07:03:48.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.049+0000 7fb28dc00700 1 Processor -- start 2024-05-27T07:03:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.049+0000 7fb28dc00700 1 -- start start 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 0x7fb2881a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 0x7fb2881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288108690 0x7fb2881a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb28811a910 con 0x7fb288103240 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb28811a790 con 0x7fb288103b90 2024-05-27T07:03:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.050+0000 7fb28dc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb28811aa90 con 0x7fb288108690 2024-05-27T07:03:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.051+0000 7fb286000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 0x7fb2881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.051+0000 7fb287400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288108690 0x7fb2881a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.051+0000 7fb286000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 0x7fb2881a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34118/0 (socket says 172.21.15.16:34118) 2024-05-27T07:03:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.051+0000 7fb286000700 1 -- 172.21.15.16:0/250856309 learned_addr learned my addr 172.21.15.16:0/250856309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.051+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 0x7fb2881a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 -- 172.21.15.16:0/250856309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288108690 msgr2=0x7fb2881a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288108690 0x7fb2881a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 -- 172.21.15.16:0/250856309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 msgr2=0x7fb2881a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 0x7fb2881a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 -- 172.21.15.16:0/250856309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2881a6180 con 0x7fb288103240 2024-05-27T07:03:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.052+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 0x7fb2881a1300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d950 tx=0x7fb27800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.053+0000 7fb27f400700 1 -- 172.21.15.16:0/250856309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278016070 con 0x7fb288103240 2024-05-27T07:03:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.053+0000 7fb27f400700 1 -- 172.21.15.16:0/250856309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb27800fd20 con 0x7fb288103240 2024-05-27T07:03:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.053+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881a6410 con 0x7fb288103240 2024-05-27T07:03:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.053+0000 7fb27f400700 1 -- 172.21.15.16:0/250856309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27800e3b0 con 0x7fb288103240 2024-05-27T07:03:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.055+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881add90 con 0x7fb288103240 2024-05-27T07:03:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.059+0000 7fb27f400700 1 -- 172.21.15.16:0/250856309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb278019070 con 0x7fb288103240 2024-05-27T07:03:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.061+0000 7fb29c81ab80 1 Processor -- start 2024-05-27T07:03:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.061+0000 7fb29c81ab80 1 -- start start 2024-05-27T07:03:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 0x5624b238a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b238ad30 0x5624b2384870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624b22b2e40 con 0x5624b238ad30 2024-05-27T07:03:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624b22a29e0 con 0x5624b2205e40 2024-05-27T07:03:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.062+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5624b22a6a20 con 0x5624b2384db0 2024-05-27T07:03:48.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb286000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb287400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 0x5624b238a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb286000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:54020/0 (socket says 172.21.15.16:54020) 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb286000700 1 -- 172.21.15.16:0/1079838323 learned_addr learned my addr 172.21.15.16:0/1079838323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb286a00700 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b238ad30 0x5624b2384870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb287400700 1 -- 172.21.15.16:0/1079838323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 msgr2=0x5624b23851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.063+0000 7fb287400700 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb287400700 1 -- 172.21.15.16:0/1079838323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b238ad30 msgr2=0x5624b2384870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb287400700 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b238ad30 0x5624b2384870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb287400700 1 -- 172.21.15.16:0/1079838323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b226b420 con 0x5624b2205e40 2024-05-27T07:03:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb286000700 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:48.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb27f400700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 0x7fb26c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb287400700 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 0x5624b238a7f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb280009fd0 tx=0x7fb2800090d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23de5a4fa67d7192 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb27f400700 1 -- 172.21.15.16:0/250856309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb2780e1540 con 0x7fb288103240 2024-05-27T07:03:48.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.064+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 0x7fb26c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.065+0000 7fb286a00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 0x7fb26c0b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2881a6050 tx=0x7fb27800bf90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.065+0000 7fb273e00700 1 -- 172.21.15.16:0/1079838323 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280020070 con 0x5624b2205e40 2024-05-27T07:03:48.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.065+0000 7fb273e00700 1 -- 172.21.15.16:0/1079838323 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800075c0 con 0x5624b2205e40 2024-05-27T07:03:48.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.065+0000 7fb273e00700 1 -- 172.21.15.16:0/1079838323 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280005280 con 0x5624b2205e40 2024-05-27T07:03:48.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 -- 172.21.15.16:0/1079838323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 msgr2=0x5624b238a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 0x5624b238a7f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb280009fd0 tx=0x7fb2800090d0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 -- 172.21.15.16:0/1079838323 shutdown_connections 2024-05-27T07:03:48.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b23851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b2205e40 0x5624b238a7f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/1079838323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b238ad30 0x5624b2384870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.067+0000 7fb29c81ab80 1 -- 172.21.15.16:0/1079838323 >> 172.21.15.16:0/1079838323 conn(0x5624b2265fc0 msgr2=0x5624b237f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.069+0000 7fb29c81ab80 1 -- 172.21.15.16:0/1079838323 shutdown_connections 2024-05-27T07:03:48.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.069+0000 7fb29c81ab80 1 -- 172.21.15.16:0/1079838323 wait complete. 2024-05-27T07:03:48.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.070+0000 7fb29c81ab80 1 -- start start 2024-05-27T07:03:48.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.070+0000 7fb29c81ab80 1 Processor -- start 2024-05-27T07:03:48.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b238ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624b21a8fa0 con 0x5624b2205e40 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624b2243760 con 0x5624b238ad30 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5624b23917a0 con 0x5624b2384db0 2024-05-27T07:03:48.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.071+0000 7fb29c81ab80 10 client.0 _ll_register_callbacks cb 0x5624b224cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.072+0000 7fb287400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.072+0000 7fb287400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.072+0000 7fb287400700 1 -- 172.21.15.16:0/3082342189 learned_addr learned my addr 172.21.15.16:0/3082342189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb286000700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b238ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 -- 172.21.15.16:0/3082342189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 msgr2=0x5624b238ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b238ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 -- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 msgr2=0x5624b238ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb287400700 1 -- 172.21.15.16:0/3082342189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b21f28d0 con 0x5624b2205e40 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.073+0000 7fb286000700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b238ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.074+0000 7fb287400700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b040 tx=0x7fb2800099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25cab59ba7bfe49 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.074+0000 7fb270c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.075+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280020040 con 0x5624b2205e40 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.075+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb280005040 con 0x5624b2205e40 2024-05-27T07:03:48.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.075+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2800075c0 con 0x5624b2205e40 2024-05-27T07:03:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb280007d30 con 0x5624b2205e40 2024-05-27T07:03:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb29c81ab80 10 client.15912 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5624b2273060 con 0x5624b2205e40 2024-05-27T07:03:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800075c0 con 0x5624b2205e40 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb280011070 con 0x5624b2205e40 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb29c81ab80 10 client.15912 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.076+0000 7fb29c81ab80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.077+0000 7fb29c81ab80 20 client.15912 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.077+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b239e4f0 0x5624b23a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.077+0000 7fb29c81ab80 4 client.15912 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:03:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.077+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b22b4d90 con 0x5624b239e4f0 2024-05-27T07:03:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.078+0000 7fb286000700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b239e4f0 0x5624b23a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.079+0000 7fb286000700 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b239e4f0 0x5624b23a0910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822a59cacc32e518 server_cookie=6d8c613200526711 in_seq=0 out_seq=0 2024-05-27T07:03:48.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.079+0000 7fb270c00700 10 client.15912 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb270c00700 1 -- 172.21.15.16:0/3082342189 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624b22b4d90 con 0x5624b239e4f0 2024-05-27T07:03:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb270c00700 10 client.15912 handle_command_reply: tid=0 2024-05-27T07:03:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 client.15912 shutdown 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 msgr2=0x5624b238e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b040 tx=0x7fb2800099e0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 shutdown_connections 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b239e4f0 0x5624b23a0910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2205e40 0x5624b238e4d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3082342189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2384db0 0x5624b238ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 >> 172.21.15.16:0/3082342189 conn(0x5624b2265fc0 msgr2=0x5624b237f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.081+0000 7fb270c00700 0 client.15912 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.082+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 shutdown_connections 2024-05-27T07:03:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.082+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3082342189 wait complete. 2024-05-27T07:03:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.082+0000 7fb29c81ab80 20 client.15912 trim_cache size 0 max 16384 2024-05-27T07:03:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.115+0000 7fb29c81ab80 1 Processor -- start 2024-05-27T07:03:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 -- start start 2024-05-27T07:03:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b2384870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b6a30 con 0x5624b2384db0 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b8570 con 0x5624b238ad30 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.116+0000 7fb29c81ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b7100 con 0x5624b2205e40 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52116/0 (socket says 172.21.15.16:52116) 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 -- 172.21.15.16:0/550216432 learned_addr learned my addr 172.21.15.16:0/550216432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb287400700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b2384870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb287400700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 -- 172.21.15.16:0/550216432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 msgr2=0x5624b2384870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b2384870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 -- 172.21.15.16:0/550216432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 msgr2=0x5624b238d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.117+0000 7fb286000700 1 -- 172.21.15.16:0/550216432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b238b140 con 0x5624b2384db0 2024-05-27T07:03:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.118+0000 7fb286000700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb274005950 tx=0x7fb27400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e3d10ad5d415b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.118+0000 7fb287400700 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.119+0000 7fb273400700 1 -- 172.21.15.16:0/550216432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274027410 con 0x5624b2384db0 2024-05-27T07:03:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.119+0000 7fb273400700 1 -- 172.21.15.16:0/550216432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb27401d880 con 0x5624b2384db0 2024-05-27T07:03:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.119+0000 7fb273400700 1 -- 172.21.15.16:0/550216432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740108c0 con 0x5624b2384db0 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 -- 172.21.15.16:0/550216432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 msgr2=0x5624b2388c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb274005950 tx=0x7fb27400b500 comp rx=0 tx=0).stop 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 -- 172.21.15.16:0/550216432 shutdown_connections 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b238d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b2384870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/550216432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b2388c40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.120+0000 7fb29c81ab80 1 -- 172.21.15.16:0/550216432 >> 172.21.15.16:0/550216432 conn(0x5624b2265fc0 msgr2=0x5624b238e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.121+0000 7fb29c81ab80 1 -- 172.21.15.16:0/550216432 shutdown_connections 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.121+0000 7fb29c81ab80 1 -- 172.21.15.16:0/550216432 wait complete. 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.121+0000 7fb29c81ab80 1 -- start start 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.122+0000 7fb29c81ab80 1 Processor -- start 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.123+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.123+0000 7fb29c81ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b23a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb287400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb287400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:54048/0 (socket says 172.21.15.16:54048) 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb287400700 1 -- 172.21.15.16:0/3484842552 learned_addr learned my addr 172.21.15.16:0/3484842552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b23a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b7ea0 con 0x5624b2384db0 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b77d0 con 0x5624b238ad30 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881b49d0 con 0x5624b2205e40 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb29c81ab80 10 client.0 _ll_register_callbacks cb 0x5624b224cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.124+0000 7fb286000700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b23a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 -- 172.21.15.16:0/3484842552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 msgr2=0x5624b23a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 -- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 msgr2=0x5624b23a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b23a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b2407d60 con 0x5624b2384db0 2024-05-27T07:03:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb287400700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:03:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb286000700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b23a1b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5624b22af0c0 tx=0x7fb27401d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480461570f188fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.126+0000 7fb270c00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:03:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27401d670 con 0x5624b2384db0 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb27401da30 con 0x5624b2384db0 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27401d880 con 0x5624b2384db0 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb27401dc30 con 0x5624b2384db0 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb29c81ab80 10 client.15918 fetch_fsmap learned FSMap version 31 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.127+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5624b2408f00 con 0x5624b2384db0 2024-05-27T07:03:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb27401d880 con 0x5624b2384db0 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb274030430 con 0x5624b2384db0 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 10 client.15918 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 20 client.15918 populate_metadata read hostname 'smithi016' 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b2409530 0x5624b240b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 4 client.15918 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.128+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624b22ae570 con 0x5624b2409530 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.129+0000 7fb287400700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b2409530 0x5624b240b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:48.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.130+0000 7fb287400700 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b2409530 0x5624b240b950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69cc4b161fb2cd9 server_cookie=2c9ba7088b785303 in_seq=0 out_seq=0 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.130+0000 7fb270c00700 10 client.15918 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb270c00700 1 -- 172.21.15.16:0/3484842552 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5624b22ae570 con 0x5624b2409530 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb270c00700 10 client.15918 handle_command_reply: tid=0 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 client.15918 shutdown 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 msgr2=0x5624b23a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b23a1b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5624b22af0c0 tx=0x7fb27401d550 comp rx=0 tx=0).stop 2024-05-27T07:03:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 shutdown_connections 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5624b2409530 0x5624b240b950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624b238ad30 0x5624b23a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624b2205e40 0x5624b23a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 --2- 172.21.15.16:0/3484842552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b2384db0 0x5624b23a1b90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.131+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 >> 172.21.15.16:0/3484842552 conn(0x5624b2265fc0 msgr2=0x5624b238e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.132+0000 7fb270c00700 0 client.15918 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:03:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.132+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 shutdown_connections 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.132+0000 7fb29c81ab80 1 -- 172.21.15.16:0/3484842552 wait complete. 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:03:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.133+0000 7fb29c81ab80 20 client.15918 trim_cache size 0 max 16384 2024-05-27T07:03:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 msgr2=0x7fb26c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 0x7fb26c0b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2881a6050 tx=0x7fb27800bf90 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 msgr2=0x7fb2881a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 0x7fb2881a1300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb27800d950 tx=0x7fb27800dc60 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 shutdown_connections 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb26c0b2a70 0x7fb26c0b4f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288108690 0x7fb2881a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb288103240 0x7fb2881a1300 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 --2- 172.21.15.16:0/250856309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb288103b90 0x7fb2881a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.134+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 >> 172.21.15.16:0/250856309 conn(0x7fb2880fedb0 msgr2=0x7fb288105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.135+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 shutdown_connections 2024-05-27T07:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:48.135+0000 7fb28dc00700 1 -- 172.21.15.16:0/250856309 wait complete. 2024-05-27T07:03:49.442 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:03:49.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:03:49.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:49 smithi164 ceph-mon[35327]: pgmap v90: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:03:49.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:49 smithi016 ceph-mon[31415]: pgmap v90: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:03:49.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:49 smithi069 ceph-mon[35276]: pgmap v90: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:03:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:03:51.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:51 smithi016 ceph-mon[31415]: pgmap v91: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 130 MiB/s wr, 78 op/s 2024-05-27T07:03:51.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:51 smithi069 ceph-mon[35276]: pgmap v91: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 130 MiB/s wr, 78 op/s 2024-05-27T07:03:52.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:51 smithi164 ceph-mon[35327]: pgmap v91: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 130 MiB/s wr, 78 op/s 2024-05-27T07:03:53.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:53 smithi016 ceph-mon[31415]: pgmap v92: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:53.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:53 smithi069 ceph-mon[35276]: pgmap v92: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:54.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:53 smithi164 ceph-mon[35327]: pgmap v92: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:55.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:55 smithi016 ceph-mon[31415]: pgmap v93: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:55.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:55 smithi069 ceph-mon[35276]: pgmap v93: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:55 smithi164 ceph-mon[35327]: pgmap v93: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:03:57.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:57 smithi016 ceph-mon[31415]: pgmap v94: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 128 MiB/s wr, 77 op/s 2024-05-27T07:03:57.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:57 smithi069 ceph-mon[35276]: pgmap v94: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 128 MiB/s wr, 77 op/s 2024-05-27T07:03:57.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:57 smithi164 ceph-mon[35327]: pgmap v94: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 128 MiB/s wr, 77 op/s 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 -- 172.21.15.16:0/2930792323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb201138f0 msgr2=0x7fbb20113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 --2- 172.21.15.16:0/2930792323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb201138f0 0x7fbb20113d00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0099c0 tx=0x7fbb0c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 -- 172.21.15.16:0/2930792323 shutdown_connections 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 --2- 172.21.15.16:0/2930792323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb20114240 0x7fbb2011bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 --2- 172.21.15.16:0/2930792323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb2010efc0 0x7fbb2010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 --2- 172.21.15.16:0/2930792323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb201138f0 0x7fbb20113d00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.937+0000 7fbb27200700 1 -- 172.21.15.16:0/2930792323 >> 172.21.15.16:0/2930792323 conn(0x7fbb2010ab50 msgr2=0x7fbb2010cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.938+0000 7fbb27200700 1 -- 172.21.15.16:0/2930792323 shutdown_connections 2024-05-27T07:03:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.939+0000 7fbb27200700 1 -- 172.21.15.16:0/2930792323 wait complete. 2024-05-27T07:03:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.941+0000 7fbb27200700 1 Processor -- start 2024-05-27T07:03:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.942+0000 7fbb27200700 1 -- start start 2024-05-27T07:03:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb2010efc0 0x7fbb201ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 0x7fbb201a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2011ed40 con 0x7fbb20114240 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2011ebc0 con 0x7fbb2010efc0 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb27200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2011eec0 con 0x7fbb201138f0 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:35376/0 (socket says 172.21.15.16:35376) 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb1f400700 1 -- 172.21.15.16:0/2663586089 learned_addr learned my addr 172.21.15.16:0/2663586089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:03:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.944+0000 7fbb1fe00700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb2010efc0 0x7fbb201ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb25200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 0x7fbb201a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb1f400700 1 -- 172.21.15.16:0/2663586089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb2010efc0 msgr2=0x7fbb201ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb1f400700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb2010efc0 0x7fbb201ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb1f400700 1 -- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 msgr2=0x7fbb201a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb1f400700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 0x7fbb201a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb1f400700 1 -- 172.21.15.16:0/2663586089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a5d90 con 0x7fbb201138f0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.945+0000 7fbb25200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 0x7fbb201a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.946+0000 7fbb1f400700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00b580 tx=0x7fbb0c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.947+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c009e60 con 0x7fbb201138f0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.947+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a6020 con 0x7fbb201138f0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.947+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2006f610 con 0x7fbb201138f0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.947+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb0c0047f0 con 0x7fbb201138f0 2024-05-27T07:03:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.947+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003e60 con 0x7fbb201138f0 2024-05-27T07:03:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.948+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb00005330 con 0x7fbb201138f0 2024-05-27T07:03:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.955+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbb0c0039d0 con 0x7fbb201138f0 2024-05-27T07:03:57.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.957+0000 7fbb1cc00700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 0x7fbb040b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:03:57.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.957+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fbb0c0e3e10 con 0x7fbb201138f0 2024-05-27T07:03:57.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.957+0000 7fbb1fe00700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 0x7fbb040b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:03:57.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.959+0000 7fbb1fe00700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 0x7fbb040b5000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005950 tx=0x7fbb100058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:03:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:57.969+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb0c02f070 con 0x7fbb201138f0 2024-05-27T07:03:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.453+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb00005620 con 0x7fbb201138f0 2024-05-27T07:03:58.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.455+0000 7fbb1cc00700 1 -- 172.21.15.16:0/2663586089 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7fbb0c013070 con 0x7fbb201138f0 2024-05-27T07:03:58.449 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:03:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:03:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 msgr2=0x7fbb040b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 0x7fbb040b5000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005950 tx=0x7fbb100058e0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 msgr2=0x7fbb201aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:03:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00b580 tx=0x7fbb0c0045e0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 shutdown_connections 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb040b2af0 0x7fbb040b5000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20114240 0x7fbb201a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb2010efc0 0x7fbb201ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 --2- 172.21.15.16:0/2663586089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb201138f0 0x7fbb201aba10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 >> 172.21.15.16:0/2663586089 conn(0x7fbb2010ab50 msgr2=0x7fbb201151a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.459+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 shutdown_connections 2024-05-27T07:03:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:03:58.460+0000 7fbb27200700 1 -- 172.21.15.16:0/2663586089 wait complete. 2024-05-27T07:03:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:03:58.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:58 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2663586089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:58.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:58 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2663586089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:59.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:58 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2663586089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:03:59.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:03:59.621 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:03:59 smithi016 ceph-mon[31415]: pgmap v95: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:00.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:03:59 smithi164 ceph-mon[35327]: pgmap v95: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:00.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:03:59 smithi069 ceph-mon[35276]: pgmap v95: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:02.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:01 smithi164 ceph-mon[35327]: pgmap v96: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 125 MiB/s wr, 75 op/s 2024-05-27T07:04:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:01 smithi016 ceph-mon[31415]: pgmap v96: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 125 MiB/s wr, 75 op/s 2024-05-27T07:04:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:01 smithi069 ceph-mon[35276]: pgmap v96: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 125 MiB/s wr, 75 op/s 2024-05-27T07:04:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 -- 172.21.15.16:0/2133947780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 msgr2=0x7fa1fc10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 --2- 172.21.15.16:0/2133947780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc10ad60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009990 tx=0x7fa1e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 -- 172.21.15.16:0/2133947780 shutdown_connections 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 --2- 172.21.15.16:0/2133947780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 --2- 172.21.15.16:0/2133947780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc110980 0x7fa1fc110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 --2- 172.21.15.16:0/2133947780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc10ad60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 -- 172.21.15.16:0/2133947780 >> 172.21.15.16:0/2133947780 conn(0x7fa1fc076d20 msgr2=0x7fa1fc077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 -- 172.21.15.16:0/2133947780 shutdown_connections 2024-05-27T07:04:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.862+0000 7fa202c00700 1 -- 172.21.15.16:0/2133947780 wait complete. 2024-05-27T07:04:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.863+0000 7fa202c00700 1 Processor -- start 2024-05-27T07:04:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.864+0000 7fa202c00700 1 -- start start 2024-05-27T07:04:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc110980 0x7fa1fc1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc11aa00 con 0x7fa1fc10a950 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc11a880 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.865+0000 7fa202c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc11ab80 con 0x7fa1fc110980 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa200c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc110980 0x7fa1fc1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fb400700 1 -- 172.21.15.16:0/1165261862 learned_addr learned my addr 172.21.15.16:0/1165261862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fb400700 1 -- 172.21.15.16:0/1165261862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc110980 msgr2=0x7fa1fc1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.866+0000 7fa1fb400700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc110980 0x7fa1fc1a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.867+0000 7fa1fb400700 1 -- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 msgr2=0x7fa1fc1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.867+0000 7fa1fb400700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.867+0000 7fa1fb400700 1 -- 172.21.15.16:0/1165261862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc1a6300 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.867+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.867+0000 7fa1fb400700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80057d0 tx=0x7fa1e8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.868+0000 7fa1f8c00700 1 -- 172.21.15.16:0/1165261862 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8027070 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.868+0000 7fa1f8c00700 1 -- 172.21.15.16:0/1165261862 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1e8003ef0 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.868+0000 7fa1f8c00700 1 -- 172.21.15.16:0/1165261862 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8022b00 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.869+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc1a6590 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.870+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a69f0 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.876+0000 7fa1f8c00700 1 -- 172.21.15.16:0/1165261862 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa1e8003a60 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.879+0000 7fa1f8c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 0x7fa1e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.879+0000 7fa1f8c00700 1 -- 172.21.15.16:0/1165261862 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa1e80e3cc0 con 0x7fa1fc10b2a0 2024-05-27T07:04:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.879+0000 7fa2117bfb80 1 Processor -- start 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.879+0000 7fa2117bfb80 1 -- start start 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 0x7fa1e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c19f8c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c19ebae40 con 0x564c19e0de40 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564c19eaa9e0 con 0x564c19f92d30 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.880+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c19eaea20 con 0x564c19f8cdb0 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.881+0000 7fa200c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.881+0000 7fa200c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52714/0 (socket says 172.21.15.16:52714) 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.881+0000 7fa200c00700 1 -- 172.21.15.16:0/1428047782 learned_addr learned my addr 172.21.15.16:0/1428047782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.882+0000 7fa200c00700 1 -- 172.21.15.16:0/1428047782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 msgr2=0x564c19f8c870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:04:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.882+0000 7fa200c00700 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c19f8c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.882+0000 7fa200c00700 1 -- 172.21.15.16:0/1428047782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 msgr2=0x564c19f927f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:04:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.883+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.883+0000 7fa200c00700 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f927f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.883+0000 7fa200c00700 1 -- 172.21.15.16:0/1428047782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c19e73420 con 0x564c19f8cdb0 2024-05-27T07:04:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.884+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.885+0000 7fa200c00700 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400a430 tx=0x7fa1f4009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b053d097dda8a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.886+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 0x7fa1e00b4f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x564c19f8db30 tx=0x7fa1ec009590 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.886+0000 7fa1f1600700 1 -- 172.21.15.16:0/1428047782 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4013590 con 0x564c19f8cdb0 2024-05-27T07:04:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.886+0000 7fa1f1600700 1 -- 172.21.15.16:0/1428047782 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f4019070 con 0x564c19f8cdb0 2024-05-27T07:04:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.886+0000 7fa1f1600700 1 -- 172.21.15.16:0/1428047782 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4014a50 con 0x564c19f8cdb0 2024-05-27T07:04:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.887+0000 7fa2117bfb80 1 -- 172.21.15.16:0/1428047782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 msgr2=0x564c19f8d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.887+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400a430 tx=0x7fa1f4009650 comp rx=0 tx=0).stop 2024-05-27T07:04:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 -- 172.21.15.16:0/1428047782 shutdown_connections 2024-05-27T07:04:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f8d1c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/1428047782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c19f8c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 -- 172.21.15.16:0/1428047782 >> 172.21.15.16:0/1428047782 conn(0x564c19e6dfc0 msgr2=0x564c19f878d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 -- 172.21.15.16:0/1428047782 shutdown_connections 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.888+0000 7fa2117bfb80 1 -- 172.21.15.16:0/1428047782 wait complete. 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.889+0000 7fa2117bfb80 1 -- start start 2024-05-27T07:04:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.890+0000 7fa2117bfb80 1 Processor -- start 2024-05-27T07:04:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 0x564c19f96b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c19db0fa0 con 0x564c19e0de40 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564c19e4b760 con 0x564c19f8cdb0 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c19f99800 con 0x564c19f92d30 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.891+0000 7fa2117bfb80 10 client.0 _ll_register_callbacks cb 0x564c19e54c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.892+0000 7fa200c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.892+0000 7fa200c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52728/0 (socket says 172.21.15.16:52728) 2024-05-27T07:04:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.892+0000 7fa200c00700 1 -- 172.21.15.16:0/2349202673 learned_addr learned my addr 172.21.15.16:0/2349202673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.892+0000 7fa1fb400700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 0x564c19f96b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.892+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 -- 172.21.15.16:0/2349202673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 msgr2=0x564c19f96b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 0x564c19f96b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 -- 172.21.15.16:0/2349202673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 msgr2=0x564c19f965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 -- 172.21.15.16:0/2349202673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c19dfa8d0 con 0x564c19f92d30 2024-05-27T07:04:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa1fb400700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 0x564c19f96b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa200c00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004b10 tx=0x7fa1f40137c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf9d3bf3794a06b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.893+0000 7fa1e6000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:04:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.894+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40145f0 con 0x564c19f92d30 2024-05-27T07:04:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.894+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa1f4014790 con 0x564c19f92d30 2024-05-27T07:04:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.894+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f4019070 con 0x564c19f92d30 2024-05-27T07:04:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.895+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa1f4030050 con 0x564c19f92d30 2024-05-27T07:04:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.895+0000 7fa2117bfb80 10 client.25594 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.895+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c19e7b060 con 0x564c19f92d30 2024-05-27T07:04:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f4019070 con 0x564c19f92d30 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f401d7d0 con 0x564c19f92d30 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 10 client.25594 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 10 client.25594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 20 client.25594 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c19fa66a0 0x564c19fa8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 4 client.25594 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.896+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c19ebcd90 con 0x564c19fa66a0 2024-05-27T07:04:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.897+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c19fa66a0 0x564c19fa8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.899+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c19fa66a0 0x564c19fa8ac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd2ef9e479779ceb server_cookie=d31a517c7d51ac84 in_seq=0 out_seq=0 2024-05-27T07:04:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.899+0000 7fa1e6000700 10 client.25594 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.901+0000 7fa1e6000700 1 -- 172.21.15.16:0/2349202673 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564c19ebcd90 con 0x564c19fa66a0 2024-05-27T07:04:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.901+0000 7fa1e6000700 10 client.25594 handle_command_reply: tid=0 2024-05-27T07:04:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.901+0000 7fa2117bfb80 1 client.25594 shutdown 2024-05-27T07:04:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 msgr2=0x564c19f97070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004b10 tx=0x7fa1f40137c0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 shutdown_connections 2024-05-27T07:04:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c19fa66a0 0x564c19fa8ac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f92d30 0x564c19f97070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19f965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2349202673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f8cdb0 0x564c19f96b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 >> 172.21.15.16:0/2349202673 conn(0x564c19e6dfc0 msgr2=0x564c19f87580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa1e6000700 0 client.25594 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.902+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 shutdown_connections 2024-05-27T07:04:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.903+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2349202673 wait complete. 2024-05-27T07:04:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.903+0000 7fa2117bfb80 20 client.25594 trim_cache size 0 max 16384 2024-05-27T07:04:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.931+0000 7fa2117bfb80 1 Processor -- start 2024-05-27T07:04:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.931+0000 7fa2117bfb80 1 -- start start 2024-05-27T07:04:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 0x564c19f98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b6b00 con 0x564c19f92d30 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b8640 con 0x564c19e0de40 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b71d0 con 0x564c19f8cdb0 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa1fb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f95910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51424/0 (socket says 172.21.15.16:51424) 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.932+0000 7fa1fbe00700 1 -- 172.21.15.16:0/2362159758 learned_addr learned my addr 172.21.15.16:0/2362159758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 0x564c19f98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 -- 172.21.15.16:0/2362159758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 msgr2=0x564c19f95910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f95910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 -- 172.21.15.16:0/2362159758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 msgr2=0x564c19f8c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa200c00700 1 -- 172.21.15.16:0/2362159758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c19f99330 con 0x564c19f92d30 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.933+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.934+0000 7fa200c00700 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 0x564c19f98a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa1f401dc20 tx=0x7fa1f4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2251b3f6bab5cf41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.934+0000 7fa1f0c00700 1 -- 172.21.15.16:0/2362159758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401e400 con 0x564c19f92d30 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.934+0000 7fa1f0c00700 1 -- 172.21.15.16:0/2362159758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f4019040 con 0x564c19f92d30 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.934+0000 7fa1f0c00700 1 -- 172.21.15.16:0/2362159758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401e6b0 con 0x564c19f92d30 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.935+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2362159758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 msgr2=0x564c19f98a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.935+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 0x564c19f98a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa1f401dc20 tx=0x7fa1f4009650 comp rx=0 tx=0).stop 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2362159758 shutdown_connections 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19f92d30 0x564c19f98a60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19e0de40 0x564c19f8c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/2362159758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19f95910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2362159758 >> 172.21.15.16:0/2362159758 conn(0x564c19e6dfc0 msgr2=0x564c19f90210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2362159758 shutdown_connections 2024-05-27T07:04:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 -- 172.21.15.16:0/2362159758 wait complete. 2024-05-27T07:04:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.936+0000 7fa2117bfb80 1 -- start start 2024-05-27T07:04:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.937+0000 7fa2117bfb80 1 Processor -- start 2024-05-27T07:04:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.937+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.937+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19fa9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.937+0000 7fa2117bfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c1a00ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b7f70 con 0x564c19e0de40 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b78a0 con 0x564c19f92d30 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa2117bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1b4aa0 con 0x564c19f8cdb0 2024-05-27T07:04:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa2117bfb80 10 client.0 _ll_register_callbacks cb 0x564c19f92200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38886/0 (socket says 172.21.15.16:38886) 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 -- 172.21.15.16:0/3191374993 learned_addr learned my addr 172.21.15.16:0/3191374993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa200c00700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c1a00ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 -- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 msgr2=0x564c19fa9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19fa9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 -- 172.21.15.16:0/3191374993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 msgr2=0x564c1a00ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.938+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c1a00ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.939+0000 7fa1fbe00700 1 -- 172.21.15.16:0/3191374993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c19f89030 con 0x564c19e0de40 2024-05-27T07:04:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.939+0000 7fa1fbe00700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec027fd0 tx=0x7fa1ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e251d14f4ca0c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.939+0000 7fa1e6000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.939+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec017040 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.939+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa1ec004220 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.940+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f4019040 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.940+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa1ec004420 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.940+0000 7fa2117bfb80 10 client.15936 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.940+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c1a010210 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.940+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f4019040 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa1ec01d040 con 0x564c19e0de40 2024-05-27T07:04:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 10 client.15936 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:04:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 20 client.15936 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c1a010970 0x564c1a012dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 4 client.15936 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c19eb6570 con 0x564c1a010970 2024-05-27T07:04:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.941+0000 7fa1fb400700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c1a010970 0x564c1a012dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.942+0000 7fa1fb400700 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c1a010970 0x564c1a012dd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7601b45c7bcb3987 server_cookie=13274299206de086 in_seq=0 out_seq=0 2024-05-27T07:04:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.942+0000 7fa1e6000700 10 client.15936 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa1e6000700 1 -- 172.21.15.16:0/3191374993 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c19eb6570 con 0x564c1a010970 2024-05-27T07:04:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa1e6000700 10 client.15936 handle_command_reply: tid=0 2024-05-27T07:04:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 client.15936 shutdown 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 msgr2=0x564c19fa8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec027fd0 tx=0x7fa1ec004750 comp rx=0 tx=0).stop 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 shutdown_connections 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x564c1a010970 0x564c1a012dd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c19f92d30 0x564c1a00ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c19e0de40 0x564c19fa8b20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 --2- 172.21.15.16:0/3191374993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c19f8cdb0 0x564c19fa9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa1e6000700 0 client.15936 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.943+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 >> 172.21.15.16:0/3191374993 conn(0x564c19e6dfc0 msgr2=0x564c19f95670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.945+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 shutdown_connections 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.945+0000 7fa2117bfb80 1 -- 172.21.15.16:0/3191374993 wait complete. 2024-05-27T07:04:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.946+0000 7fa2117bfb80 20 client.15936 trim_cache size 0 max 16384 2024-05-27T07:04:02.940 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 msgr2=0x7fa1e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 0x7fa1e00b4f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x564c19f8db30 tx=0x7fa1ec009590 comp rx=0 tx=0).stop 2024-05-27T07:04:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 msgr2=0x7fa1fc1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80057d0 tx=0x7fa1e8005800 comp rx=0 tx=0).stop 2024-05-27T07:04:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 shutdown_connections 2024-05-27T07:04:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa1e00b2a70 0x7fa1e00b4f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1fc110980 0x7fa1fc1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc10a950 0x7fa1fc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 --2- 172.21.15.16:0/1165261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1fc10b2a0 0x7fa1fc1a1a70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 >> 172.21.15.16:0/1165261862 conn(0x7fa1fc076d20 msgr2=0x7fa1fc108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 shutdown_connections 2024-05-27T07:04:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:02.947+0000 7fa202c00700 1 -- 172.21.15.16:0/1165261862 wait complete. 2024-05-27T07:04:03.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:03 smithi016 ceph-mon[31415]: pgmap v97: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:04.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:03 smithi164 ceph-mon[35327]: pgmap v97: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:04.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:03 smithi069 ceph-mon[35276]: pgmap v97: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:04.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:04:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:05.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:05 smithi016 ceph-mon[31415]: pgmap v98: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:06.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:05 smithi164 ceph-mon[35327]: pgmap v98: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:06.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:05 smithi069 ceph-mon[35276]: pgmap v98: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 81 MiB/s wr, 49 op/s 2024-05-27T07:04:07.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.304+0000 7fee17800700 1 -- 172.21.15.16:0/1301288210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee10078df0 msgr2=0x7fee10079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.304+0000 7fee17800700 1 --2- 172.21.15.16:0/1301288210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee10078df0 0x7fee10079200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fee00009990 tx=0x7fee00009ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 -- 172.21.15.16:0/1301288210 shutdown_connections 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 --2- 172.21.15.16:0/1301288210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101151f0 0x7fee10117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 --2- 172.21.15.16:0/1301288210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee10078df0 0x7fee10079200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 --2- 172.21.15.16:0/1301288210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 0x7fee10114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 -- 172.21.15.16:0/1301288210 >> 172.21.15.16:0/1301288210 conn(0x7fee10102070 msgr2=0x7fee101044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.305+0000 7fee17800700 1 -- 172.21.15.16:0/1301288210 shutdown_connections 2024-05-27T07:04:07.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.306+0000 7fee17800700 1 -- 172.21.15.16:0/1301288210 wait complete. 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.307+0000 7fee17800700 1 Processor -- start 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.307+0000 7fee17800700 1 -- start start 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 0x7fee101a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 0x7fee101a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee1011a980 con 0x7fee101151f0 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee1011a800 con 0x7fee10079740 2024-05-27T07:04:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee17800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee1011ab00 con 0x7fee101a5ca0 2024-05-27T07:04:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.308+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 0x7fee101a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee15800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee14e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 0x7fee101a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee15800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52758/0 (socket says 172.21.15.16:52758) 2024-05-27T07:04:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee15800700 1 -- 172.21.15.16:0/1925027553 learned_addr learned my addr 172.21.15.16:0/1925027553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee15800700 1 -- 172.21.15.16:0/1925027553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 msgr2=0x7fee101a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.309+0000 7fee15800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 0x7fee101a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.310+0000 7fee15800700 1 -- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 msgr2=0x7fee101a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.310+0000 7fee15800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 0x7fee101a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.310+0000 7fee15800700 1 -- 172.21.15.16:0/1925027553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101a67f0 con 0x7fee101a5ca0 2024-05-27T07:04:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.310+0000 7fee0fe00700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 0x7fee101a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.310+0000 7fee15800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee08007ee0 tx=0x7fee0800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.311+0000 7fee0d600700 1 -- 172.21.15.16:0/1925027553 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0800dbd0 con 0x7fee101a5ca0 2024-05-27T07:04:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.311+0000 7fee0d600700 1 -- 172.21.15.16:0/1925027553 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee0800dd70 con 0x7fee101a5ca0 2024-05-27T07:04:07.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.312+0000 7fee0d600700 1 -- 172.21.15.16:0/1925027553 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee080058d0 con 0x7fee101a5ca0 2024-05-27T07:04:07.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.313+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101a6ae0 con 0x7fee101a5ca0 2024-05-27T07:04:07.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.313+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101ae210 con 0x7fee101a5ca0 2024-05-27T07:04:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.319+0000 7fee26280b80 1 Processor -- start 2024-05-27T07:04:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.321+0000 7fee0d600700 1 -- 172.21.15.16:0/1925027553 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fee0801a020 con 0x7fee101a5ca0 2024-05-27T07:04:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.321+0000 7fee26280b80 1 -- start start 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.321+0000 7fee26280b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee26280b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0fee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee26280b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ffb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee26280b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5596d0f1ce40 con 0x5596d0f5b9f0 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee26280b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5596d0f0c9e0 con 0x5596d0feee70 2024-05-27T07:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee26280b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5596d0f10a20 con 0x5596d0e6fe40 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.322+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38922/0 (socket says 172.21.15.16:38922) 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0fe00700 1 -- 172.21.15.16:0/3604911562 learned_addr learned my addr 172.21.15.16:0/3604911562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee14e00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0fee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0d600700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 0x7fedf80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0fe00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ffb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0d600700 1 -- 172.21.15.16:0/1925027553 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fee080e1ad0 con 0x7fee101a5ca0 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0fe00700 1 -- 172.21.15.16:0/3604911562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 msgr2=0x5596d0fee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.323+0000 7fee0fe00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0fee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee0fe00700 1 -- 172.21.15.16:0/3604911562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 msgr2=0x5596d0ffb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee0fe00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ffb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee15800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 0x7fedf80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee0fe00700 1 -- 172.21.15.16:0/3604911562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d0ed5420 con 0x5596d0f5b9f0 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee14e00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0fee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fee0fe00700 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fee04005e20 tx=0x7fee04008420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bfd9d2c9ecac7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.324+0000 7fedfe000700 1 -- 172.21.15.16:0/3604911562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04004a30 con 0x5596d0f5b9f0 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.325+0000 7fedfe000700 1 -- 172.21.15.16:0/3604911562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee040054b0 con 0x5596d0f5b9f0 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.325+0000 7fedfe000700 1 -- 172.21.15.16:0/3604911562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04011810 con 0x5596d0f5b9f0 2024-05-27T07:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 -- 172.21.15.16:0/3604911562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 msgr2=0x5596d0fee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee15800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 0x7fedf80b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee08009440 tx=0x7fee08009390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fee04005e20 tx=0x7fee04008420 comp rx=0 tx=0).stop 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 -- 172.21.15.16:0/3604911562 shutdown_connections 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ffb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0f5b9f0 0x5596d0fee330 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fee04005e20 tx=0x7fee04008420 comp rx=0 tx=0).stop 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 --2- 172.21.15.16:0/3604911562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0fee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 -- 172.21.15.16:0/3604911562 >> 172.21.15.16:0/3604911562 conn(0x5596d0f09390 msgr2=0x5596d0fea4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.326+0000 7fee26280b80 1 -- 172.21.15.16:0/3604911562 shutdown_connections 2024-05-27T07:04:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.327+0000 7fee26280b80 1 -- 172.21.15.16:0/3604911562 wait complete. 2024-05-27T07:04:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.327+0000 7fee26280b80 1 -- start start 2024-05-27T07:04:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.328+0000 7fee26280b80 1 Processor -- start 2024-05-27T07:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.329+0000 7fee26280b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0ff8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.329+0000 7fee26280b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ff8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.329+0000 7fee26280b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee26280b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5596d0e12fa0 con 0x5596d0ff9190 2024-05-27T07:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee26280b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5596d0ead760 con 0x5596d0feee70 2024-05-27T07:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0ff8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0ff8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52774/0 (socket says 172.21.15.16:52774) 2024-05-27T07:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee0fe00700 1 -- 172.21.15.16:0/525635119 learned_addr learned my addr 172.21.15.16:0/525635119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee14e00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ff8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.330+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5596d100cb90 con 0x5596d0e6fe40 2024-05-27T07:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.331+0000 7fee0fe00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.331+0000 7fee26280b80 10 client.0 _ll_register_callbacks cb 0x5596d0eb6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.331+0000 7fee14e00700 1 -- 172.21.15.16:0/525635119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 msgr2=0x5596d0ff8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.331+0000 7fee14e00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0ff8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.331+0000 7fee14e00700 1 -- 172.21.15.16:0/525635119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 msgr2=0x5596d1015ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.332+0000 7fee14e00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.332+0000 7fee14e00700 1 -- 172.21.15.16:0/525635119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d0e5c8d0 con 0x5596d0feee70 2024-05-27T07:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.332+0000 7fee0fe00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.333+0000 7fee14e00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ff8c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee00008000 tx=0x7fee00005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f04b314feffbf9be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.333+0000 7fedf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.333+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00028060 con 0x5596d0feee70 2024-05-27T07:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.333+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee040054b0 con 0x5596d0feee70 2024-05-27T07:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.333+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fee000039c0 con 0x5596d0feee70 2024-05-27T07:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.334+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee00003bc0 con 0x5596d0feee70 2024-05-27T07:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.335+0000 7fee26280b80 10 client.35675 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.335+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5596d0edd060 con 0x5596d0feee70 2024-05-27T07:04:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.336+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee040054b0 con 0x5596d0feee70 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fee0000f4d0 con 0x5596d0feee70 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 10 client.35675 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 10 client.35675 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 20 client.35675 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d101a040 0x5596d101c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 4 client.35675 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.337+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d0f1ed90 con 0x5596d101a040 2024-05-27T07:04:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.338+0000 7fee0fe00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d101a040 0x5596d101c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.339+0000 7fee0fe00700 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d101a040 0x5596d101c480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=324ebfa256d44360 server_cookie=1da1a3c8c7899326 in_seq=0 out_seq=0 2024-05-27T07:04:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.339+0000 7fedf2a00700 10 client.35675 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.341+0000 7fedf2a00700 1 -- 172.21.15.16:0/525635119 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5596d0f1ed90 con 0x5596d101a040 2024-05-27T07:04:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.341+0000 7fedf2a00700 10 client.35675 handle_command_reply: tid=0 2024-05-27T07:04:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.341+0000 7fee26280b80 1 client.35675 shutdown 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 msgr2=0x5596d0ff8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ff8c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee00008000 tx=0x7fee00005790 comp rx=0 tx=0).stop 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 shutdown_connections 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0e6fe40 0x5596d0ff8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d101a040 0x5596d101c480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fedf2a00700 0 client.35675 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 --2- 172.21.15.16:0/525635119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0feee70 0x5596d0ff8c50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.342+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 >> 172.21.15.16:0/525635119 conn(0x5596d0f09390 msgr2=0x5596d0fee350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.343+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 shutdown_connections 2024-05-27T07:04:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.343+0000 7fee26280b80 1 -- 172.21.15.16:0/525635119 wait complete. 2024-05-27T07:04:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.344+0000 7fee26280b80 20 client.35675 trim_cache size 0 max 16384 2024-05-27T07:04:07.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.372+0000 7fee26280b80 1 Processor -- start 2024-05-27T07:04:07.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.372+0000 7fee26280b80 1 -- start start 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.372+0000 7fee26280b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee26280b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 0x5596d0fea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee26280b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee26280b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee101ae800 con 0x5596d0ff9190 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee26280b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b0340 con 0x5596d0e6fe40 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee26280b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee101aeed0 con 0x5596d0feee70 2024-05-27T07:04:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38944/0 (socket says 172.21.15.16:38944) 2024-05-27T07:04:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.373+0000 7fee0fe00700 1 -- 172.21.15.16:0/3540280837 learned_addr learned my addr 172.21.15.16:0/3540280837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.374+0000 7fee14e00700 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 0x5596d0fea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.374+0000 7fee0fe00700 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 -- 172.21.15.16:0/3540280837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 msgr2=0x5596d0ff3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 -- 172.21.15.16:0/3540280837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 msgr2=0x5596d1015c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 -- 172.21.15.16:0/3540280837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d0fef840 con 0x5596d0feee70 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.375+0000 7fee14e00700 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 0x5596d0fea590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee0000fa90 tx=0x7fee000098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad42fcfb5cd4921d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.376+0000 7fedfd600700 1 -- 172.21.15.16:0/3540280837 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00028040 con 0x5596d0feee70 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.376+0000 7fedfd600700 1 -- 172.21.15.16:0/3540280837 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee000037c0 con 0x5596d0feee70 2024-05-27T07:04:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.376+0000 7fedfd600700 1 -- 172.21.15.16:0/3540280837 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0000f6c0 con 0x5596d0feee70 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 -- 172.21.15.16:0/3540280837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 msgr2=0x5596d0fea590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 0x5596d0fea590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee0000fa90 tx=0x7fee000098c0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 -- 172.21.15.16:0/3540280837 shutdown_connections 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0ff9190 0x5596d1015c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 --2- 172.21.15.16:0/3540280837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0feee70 0x5596d0fea590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 -- 172.21.15.16:0/3540280837 >> 172.21.15.16:0/3540280837 conn(0x5596d0f09390 msgr2=0x5596d0fe9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.378+0000 7fee26280b80 1 -- 172.21.15.16:0/3540280837 shutdown_connections 2024-05-27T07:04:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.380+0000 7fee26280b80 1 -- 172.21.15.16:0/3540280837 wait complete. 2024-05-27T07:04:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.380+0000 7fee26280b80 1 -- start start 2024-05-27T07:04:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.380+0000 7fee26280b80 1 Processor -- start 2024-05-27T07:04:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.381+0000 7fee26280b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0ff9190 0x5596d0ff28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee101afc70 con 0x5596d0feee70 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee101af5a0 con 0x5596d0e6fe40 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee100512b0 con 0x5596d0ff9190 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee26280b80 10 client.0 _ll_register_callbacks cb 0x5596d0eb6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.382+0000 7fee14e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38948/0 (socket says 172.21.15.16:38948) 2024-05-27T07:04:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 -- 172.21.15.16:0/2128320081 learned_addr learned my addr 172.21.15.16:0/2128320081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee0fe00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0ff9190 0x5596d0ff28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 -- 172.21.15.16:0/2128320081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0ff9190 msgr2=0x5596d0ff28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0ff9190 0x5596d0ff28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 -- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 msgr2=0x5596d0ff3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.383+0000 7fee14e00700 1 -- 172.21.15.16:0/2128320081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d0ff3060 con 0x5596d0feee70 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.384+0000 7fee0fe00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.384+0000 7fee14e00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fee00005530 tx=0x7fee00005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db4293b672e0fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.384+0000 7fedf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.385+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00028040 con 0x5596d0feee70 2024-05-27T07:04:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.385+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee000037c0 con 0x5596d0feee70 2024-05-27T07:04:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.385+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fee0000b980 con 0x5596d0feee70 2024-05-27T07:04:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.385+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee00044050 con 0x5596d0feee70 2024-05-27T07:04:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.385+0000 7fee26280b80 10 client.15954 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5596d1011e60 con 0x5596d0feee70 2024-05-27T07:04:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee000037c0 con 0x5596d0feee70 2024-05-27T07:04:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fee00023430 con 0x5596d0feee70 2024-05-27T07:04:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 10 client.15954 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:04:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 20 client.15954 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d10125a0 0x5596d1086b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 4 client.15954 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.386+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596d0f18570 con 0x5596d10125a0 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.387+0000 7fee0fe00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d10125a0 0x5596d1086b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.388+0000 7fee0fe00700 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d10125a0 0x5596d1086b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d72a2918c53b196c server_cookie=337d77bf168c1253 in_seq=0 out_seq=0 2024-05-27T07:04:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.388+0000 7fedf2a00700 10 client.15954 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fedf2a00700 1 -- 172.21.15.16:0/2128320081 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596d0f18570 con 0x5596d10125a0 2024-05-27T07:04:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fedf2a00700 10 client.15954 handle_command_reply: tid=0 2024-05-27T07:04:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fee26280b80 1 client.15954 shutdown 2024-05-27T07:04:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 msgr2=0x5596d0ff2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fee00005530 tx=0x7fee00005900 comp rx=0 tx=0).stop 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.389+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 shutdown_connections 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5596d0ff9190 0x5596d0ff28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5596d10125a0 0x5596d1086b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d0e6fe40 0x5596d0ff3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 --2- 172.21.15.16:0/2128320081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d0feee70 0x5596d0ff2270 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 >> 172.21.15.16:0/2128320081 conn(0x5596d0f09390 msgr2=0x5596d0fefed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fedf2a00700 0 client.15954 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 shutdown_connections 2024-05-27T07:04:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 1 -- 172.21.15.16:0/2128320081 wait complete. 2024-05-27T07:04:07.383 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.390+0000 7fee26280b80 20 client.15954 trim_cache size 0 max 16384 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 msgr2=0x7fedf80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 0x7fedf80b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee08009440 tx=0x7fee08009390 comp rx=0 tx=0).stop 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 msgr2=0x7fee101a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee08007ee0 tx=0x7fee0800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 shutdown_connections 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fedf80b2a70 0x7fedf80b4f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee101a5ca0 0x7fee101a60b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee10079740 0x7fee101a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 --2- 172.21.15.16:0/1925027553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee101151f0 0x7fee101a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 >> 172.21.15.16:0/1925027553 conn(0x7fee10102070 msgr2=0x7fee10104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 shutdown_connections 2024-05-27T07:04:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:07.392+0000 7fee17800700 1 -- 172.21.15.16:0/1925027553 wait complete. 2024-05-27T07:04:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:07 smithi164 ceph-mon[35327]: pgmap v99: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 122 MiB/s wr, 89 op/s 2024-05-27T07:04:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:07 smithi016 ceph-mon[31415]: pgmap v99: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 122 MiB/s wr, 89 op/s 2024-05-27T07:04:08.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:07 smithi069 ceph-mon[35276]: pgmap v99: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 122 MiB/s wr, 89 op/s 2024-05-27T07:04:08.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:04:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:09.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:09 smithi016 ceph-mon[31415]: pgmap v100: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 81 MiB/s wr, 64 op/s 2024-05-27T07:04:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:09 smithi164 ceph-mon[35327]: pgmap v100: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 81 MiB/s wr, 64 op/s 2024-05-27T07:04:10.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:09 smithi069 ceph-mon[35276]: pgmap v100: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 81 MiB/s wr, 64 op/s 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 -- 172.21.15.16:0/698813056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4115170 msgr2=0x7fc2a4117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 --2- 172.21.15.16:0/698813056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4115170 0x7fc2a4117590 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009f90 tx=0x7fc29c00b330 comp rx=0 tx=0).stop 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 -- 172.21.15.16:0/698813056 shutdown_connections 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 --2- 172.21.15.16:0/698813056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4115170 0x7fc2a4117590 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 --2- 172.21.15.16:0/698813056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a4075b50 0x7fc2a4075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 --2- 172.21.15.16:0/698813056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a40764a0 0x7fc2a4114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 -- 172.21.15.16:0/698813056 >> 172.21.15.16:0/698813056 conn(0x7fc2a4079d00 msgr2=0x7fc2a407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 -- 172.21.15.16:0/698813056 shutdown_connections 2024-05-27T07:04:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.558+0000 7fc2a9400700 1 -- 172.21.15.16:0/698813056 wait complete. 2024-05-27T07:04:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.559+0000 7fc2a9400700 1 Processor -- start 2024-05-27T07:04:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.561+0000 7fc2a9400700 1 -- start start 2024-05-27T07:04:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.562+0000 7fc2a9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.562+0000 7fc2a9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a40764a0 0x7fc2a41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.562+0000 7fc2a9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a4115170 0x7fc2a41a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a411a720 con 0x7fc2a4075b50 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a9400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a411a5a0 con 0x7fc2a4115170 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a9400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a411a8a0 con 0x7fc2a40764a0 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38974/0 (socket says 172.21.15.16:38974) 2024-05-27T07:04:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a3e00700 1 -- 172.21.15.16:0/2145112760 learned_addr learned my addr 172.21.15.16:0/2145112760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a40764a0 0x7fc2a41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.563+0000 7fc2a8a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a4115170 0x7fc2a41a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.564+0000 7fc2a3e00700 1 -- 172.21.15.16:0/2145112760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a40764a0 msgr2=0x7fc2a41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.564+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a40764a0 0x7fc2a41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.564+0000 7fc2a3e00700 1 -- 172.21.15.16:0/2145112760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a4115170 msgr2=0x7fc2a41a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.564+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a4115170 0x7fc2a41a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.564+0000 7fc2a3e00700 1 -- 172.21.15.16:0/2145112760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a41a5fc0 con 0x7fc2a4075b50 2024-05-27T07:04:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.565+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e9c0 tx=0x7fc29400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.566+0000 7fc2a0c00700 1 -- 172.21.15.16:0/2145112760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29400cbd0 con 0x7fc2a4075b50 2024-05-27T07:04:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.566+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a41a62b0 con 0x7fc2a4075b50 2024-05-27T07:04:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.566+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a41adbd0 con 0x7fc2a4075b50 2024-05-27T07:04:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.568+0000 7fc2a0c00700 1 -- 172.21.15.16:0/2145112760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc29400cd70 con 0x7fc2a4075b50 2024-05-27T07:04:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.568+0000 7fc2a0c00700 1 -- 172.21.15.16:0/2145112760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294005410 con 0x7fc2a4075b50 2024-05-27T07:04:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.572+0000 7fc2a0c00700 1 -- 172.21.15.16:0/2145112760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc294021440 con 0x7fc2a4075b50 2024-05-27T07:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.575+0000 7fc2b95c6b80 1 Processor -- start 2024-05-27T07:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.575+0000 7fc2a0c00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 0x7fc2880b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.575+0000 7fc2b95c6b80 1 -- start start 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.575+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.575+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237938a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623792abe40 con 0x5623793882c0 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56237929b9e0 con 0x5623791fee40 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56237929fa20 con 0x5623792ea9f0 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a0c00700 1 -- 172.21.15.16:0/2145112760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fc2940e3a30 con 0x7fc2a4075b50 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 0x7fc2880b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a3400700 1 -- 172.21.15.16:0/84304882 learned_addr learned my addr 172.21.15.16:0/84304882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237938a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.576+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a3400700 1 -- 172.21.15.16:0/84304882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 msgr2=0x56237937e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a3400700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a3400700 1 -- 172.21.15.16:0/84304882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 msgr2=0x56237938a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a3400700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237938a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a3400700 1 -- 172.21.15.16:0/84304882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562379264420 con 0x5623791fee40 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.577+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237938a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.578+0000 7fc2a3400700 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc290009c90 tx=0x7fc2900103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5596bc635ffabda7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.578+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 0x7fc2880b5000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x562379380210 tx=0x7fc29c0097c0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.578+0000 7fc299600700 1 -- 172.21.15.16:0/84304882 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290010bf0 con 0x5623791fee40 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.579+0000 7fc299600700 1 -- 172.21.15.16:0/84304882 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc290010d90 con 0x5623791fee40 2024-05-27T07:04:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.579+0000 7fc299600700 1 -- 172.21.15.16:0/84304882 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900116a0 con 0x5623791fee40 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/84304882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 msgr2=0x56237937ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc290009c90 tx=0x7fc2900103d0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/84304882 shutdown_connections 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237938a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/84304882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x56237937ec90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/84304882 >> 172.21.15.16:0/84304882 conn(0x562379298390 msgr2=0x562379379710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.580+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/84304882 shutdown_connections 2024-05-27T07:04:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.581+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/84304882 wait complete. 2024-05-27T07:04:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.581+0000 7fc2b95c6b80 1 -- start start 2024-05-27T07:04:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 Processor -- start 2024-05-27T07:04:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x562379385740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379385c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623791a1fa0 con 0x5623793882c0 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56237923c760 con 0x5623791fee40 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.582+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56237938ae00 con 0x5623792ea9f0 2024-05-27T07:04:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2b95c6b80 10 client.0 _ll_register_callbacks cb 0x562379245c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51512/0 (socket says 172.21.15.16:51512) 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2a3e00700 1 -- 172.21.15.16:0/2179725141 learned_addr learned my addr 172.21.15.16:0/2179725141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2a3400700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x562379385740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.583+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379385c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3400700 1 -- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 msgr2=0x562379385200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3400700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3400700 1 -- 172.21.15.16:0/2179725141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 msgr2=0x562379385c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3400700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379385c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3400700 1 -- 172.21.15.16:0/2179725141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623791eb8d0 con 0x5623792ea9f0 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.584+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379385c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.585+0000 7fc2a3400700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x562379385740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc29001ce60 tx=0x7fc29001c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1502cd708226257 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.585+0000 7fc28e000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:04:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.586+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290005260 con 0x5623792ea9f0 2024-05-27T07:04:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.586+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc290010d90 con 0x5623792ea9f0 2024-05-27T07:04:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.586+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fc290005400 con 0x5623792ea9f0 2024-05-27T07:04:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.587+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc290005600 con 0x5623792ea9f0 2024-05-27T07:04:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.587+0000 7fc2b95c6b80 10 client.25627 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.587+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56237926c060 con 0x5623792ea9f0 2024-05-27T07:04:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.588+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc290010d90 con 0x5623792ea9f0 2024-05-27T07:04:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.588+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc290024070 con 0x5623792ea9f0 2024-05-27T07:04:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.589+0000 7fc2b95c6b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.589+0000 7fc2b95c6b80 10 client.25627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:04:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.589+0000 7fc2b95c6b80 20 client.25627 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.589+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379397b80 0x562379399fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.590+0000 7fc2b95c6b80 4 client.25627 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.590+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623792add90 con 0x562379397b80 2024-05-27T07:04:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.590+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379397b80 0x562379399fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.593+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379397b80 0x562379399fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65a7970ea4bfc635 server_cookie=1bd92e3e2f71b7c9 in_seq=0 out_seq=0 2024-05-27T07:04:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.593+0000 7fc28e000700 10 client.25627 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:04:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.595+0000 7fc28e000700 1 -- 172.21.15.16:0/2179725141 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5623792add90 con 0x562379397b80 2024-05-27T07:04:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.595+0000 7fc28e000700 10 client.25627 handle_command_reply: tid=0 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.595+0000 7fc2b95c6b80 1 client.25627 shutdown 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.596+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 msgr2=0x562379385740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.596+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x562379385740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc29001ce60 tx=0x7fc29001c730 comp rx=0 tx=0).stop 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.596+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 shutdown_connections 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.596+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379397b80 0x562379399fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379385c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623791fee40 0x562379385200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2179725141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623792ea9f0 0x562379385740 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 >> 172.21.15.16:0/2179725141 conn(0x562379298390 msgr2=0x562379379260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc28e000700 0 client.25627 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:04:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 shutdown_connections 2024-05-27T07:04:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2179725141 wait complete. 2024-05-27T07:04:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.597+0000 7fc2b95c6b80 20 client.25627 trim_cache size 0 max 16384 2024-05-27T07:04:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.623+0000 7fc2b95c6b80 1 Processor -- start 2024-05-27T07:04:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.623+0000 7fc2b95c6b80 1 -- start start 2024-05-27T07:04:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x562379393f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc28400ed20 con 0x5623793882c0 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc284010860 con 0x5623792ea9f0 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc28400f3f0 con 0x5623791fee40 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2a8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2a8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39012/0 (socket says 172.21.15.16:39012) 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2a8a00700 1 -- 172.21.15.16:0/1993481800 learned_addr learned my addr 172.21.15.16:0/1993481800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.624+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x562379393f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 -- 172.21.15.16:0/1993481800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 msgr2=0x562379393f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x562379393f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 -- 172.21.15.16:0/1993481800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 msgr2=0x56237937e750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:04:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 -- 172.21.15.16:0/1993481800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562379387aa0 con 0x5623793882c0 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.625+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc29c004f00 tx=0x7fc29c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89990e7d18d138c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.626+0000 7fc298c00700 1 -- 172.21.15.16:0/1993481800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c02f070 con 0x5623793882c0 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.626+0000 7fc298c00700 1 -- 172.21.15.16:0/1993481800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc29c025040 con 0x5623793882c0 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.626+0000 7fc298c00700 1 -- 172.21.15.16:0/1993481800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c024040 con 0x5623793882c0 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.627+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/1993481800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 msgr2=0x56237937ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.627+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc29c004f00 tx=0x7fc29c01f000 comp rx=0 tx=0).stop 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/1993481800 shutdown_connections 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x56237937ec90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x562379393f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/1993481800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237937e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/1993481800 >> 172.21.15.16:0/1993481800 conn(0x562379298390 msgr2=0x562379383af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/1993481800 shutdown_connections 2024-05-27T07:04:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.628+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/1993481800 wait complete. 2024-05-27T07:04:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.629+0000 7fc2b95c6b80 1 -- start start 2024-05-27T07:04:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.629+0000 7fc2b95c6b80 1 Processor -- start 2024-05-27T07:04:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379400660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc284010190 con 0x5623793882c0 2024-05-27T07:04:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc28400fac0 con 0x5623792ea9f0 2024-05-27T07:04:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc28400ccc0 con 0x5623791fee40 2024-05-27T07:04:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.630+0000 7fc2b95c6b80 10 client.0 _ll_register_callbacks cb 0x562379245c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.631+0000 7fc2a8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379400660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.631+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.632+0000 7fc2a3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.632+0000 7fc2a3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51524/0 (socket says 172.21.15.16:51524) 2024-05-27T07:04:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.632+0000 7fc2a3400700 1 -- 172.21.15.16:0/2101284608 learned_addr learned my addr 172.21.15.16:0/2101284608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.632+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52868/0 (socket says 172.21.15.16:52868) 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a8a00700 1 -- 172.21.15.16:0/2101284608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 msgr2=0x56237938b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a8a00700 1 -- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 msgr2=0x56237938b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a8a00700 1 -- 172.21.15.16:0/2101284608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237938bef0 con 0x5623793882c0 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a3400700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.633+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.634+0000 7fc2a8a00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379400660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00b6c0 tx=0x7fc29c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96913f6fd9a897c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.634+0000 7fc28e000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.635+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c02f070 con 0x5623793882c0 2024-05-27T07:04:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.635+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fc29c004040 con 0x5623793882c0 2024-05-27T07:04:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.635+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29c025040 con 0x5623793882c0 2024-05-27T07:04:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.636+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc29c045050 con 0x5623793882c0 2024-05-27T07:04:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.636+0000 7fc2b95c6b80 10 client.15972 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.636+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562379401b70 con 0x5623793882c0 2024-05-27T07:04:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.637+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc29c0046d0 con 0x5623793882c0 2024-05-27T07:04:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.637+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc29c02a040 con 0x5623793882c0 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 20 client.15972 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379402170 0x562379404590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 4 client.15972 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.638+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc29c025040 con 0x562379402170 2024-05-27T07:04:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.639+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379402170 0x562379404590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.640+0000 7fc2a3e00700 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379402170 0x562379404590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e3a99ec39b5eb1e server_cookie=a3e6030e1412d53 in_seq=0 out_seq=0 2024-05-27T07:04:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.640+0000 7fc28e000700 10 client.15972 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:04:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.641+0000 7fc28e000700 1 -- 172.21.15.16:0/2101284608 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc29c025040 con 0x562379402170 2024-05-27T07:04:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.641+0000 7fc28e000700 10 client.15972 handle_command_reply: tid=0 2024-05-27T07:04:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.641+0000 7fc2b95c6b80 1 client.15972 shutdown 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 msgr2=0x562379400660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379400660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00b6c0 tx=0x7fc29c00b330 comp rx=0 tx=0).stop 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 shutdown_connections 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x562379402170 0x562379404590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623793882c0 0x562379400660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623791fee40 0x56237938b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc28e000700 0 client.15972 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:04:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 --2- 172.21.15.16:0/2101284608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623792ea9f0 0x56237938b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.642+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 >> 172.21.15.16:0/2101284608 conn(0x562379298390 msgr2=0x56237937e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.643+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 shutdown_connections 2024-05-27T07:04:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.643+0000 7fc2b95c6b80 1 -- 172.21.15.16:0/2101284608 wait complete. 2024-05-27T07:04:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.643+0000 7fc2b95c6b80 20 client.15972 trim_cache size 0 max 16384 2024-05-27T07:04:11.636 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.644+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 msgr2=0x7fc2880b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.644+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 0x7fc2880b5000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x562379380210 tx=0x7fc29c0097c0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 msgr2=0x7fc2a41a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e9c0 tx=0x7fc29400ecd0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 shutdown_connections 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fc2880b2af0 0x7fc2880b5000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a4115170 0x7fc2a41a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a4075b50 0x7fc2a41a10f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 --2- 172.21.15.16:0/2145112760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a40764a0 0x7fc2a41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.645+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 >> 172.21.15.16:0/2145112760 conn(0x7fc2a4079d00 msgr2=0x7fc2a4100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.646+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 shutdown_connections 2024-05-27T07:04:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:11.646+0000 7fc2a9400700 1 -- 172.21.15.16:0/2145112760 wait complete. 2024-05-27T07:04:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:11 smithi016 ceph-mon[31415]: pgmap v101: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 MiB/s wr, 90 op/s 2024-05-27T07:04:12.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:11 smithi164 ceph-mon[35327]: pgmap v101: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 MiB/s wr, 90 op/s 2024-05-27T07:04:12.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:11 smithi069 ceph-mon[35276]: pgmap v101: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 MiB/s wr, 90 op/s 2024-05-27T07:04:12.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:04:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:14.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:13 smithi164 ceph-mon[35327]: pgmap v102: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:14.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:13 smithi016 ceph-mon[31415]: pgmap v102: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:14.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:13 smithi069 ceph-mon[35276]: pgmap v102: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:15.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:15 smithi016 ceph-mon[31415]: pgmap v103: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 -- 172.21.15.16:0/3047909248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4075af0 msgr2=0x7efde4075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 --2- 172.21.15.16:0/3047909248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4075af0 0x7efde4075f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009990 tx=0x7efdd4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 -- 172.21.15.16:0/3047909248 shutdown_connections 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 --2- 172.21.15.16:0/3047909248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde4110bf0 0x7efde4113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 --2- 172.21.15.16:0/3047909248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4075af0 0x7efde4075f00 secure :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009990 tx=0x7efdd4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 --2- 172.21.15.16:0/3047909248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde41106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.877+0000 7efde8a00700 1 -- 172.21.15.16:0/3047909248 >> 172.21.15.16:0/3047909248 conn(0x7efde4100030 msgr2=0x7efde4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.879+0000 7efde8a00700 1 -- 172.21.15.16:0/3047909248 shutdown_connections 2024-05-27T07:04:15.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.879+0000 7efde8a00700 1 -- 172.21.15.16:0/3047909248 wait complete. 2024-05-27T07:04:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.880+0000 7efde8a00700 1 Processor -- start 2024-05-27T07:04:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 -- start start 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4110bf0 0x7efde419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efde4116580 con 0x7efde4110bf0 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efde4116400 con 0x7efde4076440 2024-05-27T07:04:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.881+0000 7efde8a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efde4116700 con 0x7efde41a1900 2024-05-27T07:04:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.882+0000 7efde2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.882+0000 7efde2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55498/0 (socket says 172.21.15.16:55498) 2024-05-27T07:04:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.882+0000 7efde2800700 1 -- 172.21.15.16:0/3865917706 learned_addr learned my addr 172.21.15.16:0/3865917706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde1e00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde1400700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4110bf0 0x7efde419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 -- 172.21.15.16:0/3865917706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 msgr2=0x7efde419d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde419d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 -- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4110bf0 msgr2=0x7efde419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4110bf0 0x7efde419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 -- 172.21.15.16:0/3865917706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde41a2450 con 0x7efde41a1900 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde1e00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.883+0000 7efde2800700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efddc00ef90 tx=0x7efddc00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.884+0000 7efddaa00700 1 -- 172.21.15.16:0/3865917706 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0092e0 con 0x7efde41a1900 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.884+0000 7efddaa00700 1 -- 172.21.15.16:0/3865917706 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efddc004bb0 con 0x7efde41a1900 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.884+0000 7efddaa00700 1 -- 172.21.15.16:0/3865917706 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc004280 con 0x7efde41a1900 2024-05-27T07:04:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.885+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde41a26e0 con 0x7efde41a1900 2024-05-27T07:04:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.885+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41a9e70 con 0x7efde41a1900 2024-05-27T07:04:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.892+0000 7efddaa00700 1 -- 172.21.15.16:0/3865917706 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efddc01b030 con 0x7efde41a1900 2024-05-27T07:04:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.893+0000 7efdf7638b80 1 Processor -- start 2024-05-27T07:04:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.893+0000 7efdf7638b80 1 -- start start 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.894+0000 7efddaa00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 0x7efdc80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.894+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.894+0000 7efddaa00700 1 -- 172.21.15.16:0/3865917706 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7efddc0e2550 con 0x7efde41a1900 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bef61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efdf7638b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e6be23e40 con 0x562e6befbd30 2024-05-27T07:04:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efdf7638b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e6be139e0 con 0x562e6bef5db0 2024-05-27T07:04:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efdf7638b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562e6be17a20 con 0x562e6bd76e40 2024-05-27T07:04:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55514/0 (socket says 172.21.15.16:55514) 2024-05-27T07:04:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.895+0000 7efde1e00700 1 -- 172.21.15.16:0/1745875871 learned_addr learned my addr 172.21.15.16:0/1745875871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde2800700 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bef61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde1e00700 1 -- 172.21.15.16:0/1745875871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 msgr2=0x562e6bef61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde1e00700 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bef61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde1e00700 1 -- 172.21.15.16:0/1745875871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 msgr2=0x562e6bef5870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde1e00700 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef5870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.896+0000 7efde1e00700 1 -- 172.21.15.16:0/1745875871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6bddc420 con 0x562e6bd76e40 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.897+0000 7efde1400700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 0x7efdc80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.897+0000 7efde1e00700 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdd400b2e0 tx=0x7efdd4009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738306277007bca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.897+0000 7efdcf400700 1 -- 172.21.15.16:0/1745875871 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40054a0 con 0x562e6bd76e40 2024-05-27T07:04:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.898+0000 7efde1400700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 0x7efdc80b5000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005950 tx=0x7efdd00058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.898+0000 7efdcf400700 1 -- 172.21.15.16:0/1745875871 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdd4005640 con 0x562e6bd76e40 2024-05-27T07:04:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.899+0000 7efdcf400700 1 -- 172.21.15.16:0/1745875871 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400bb80 con 0x562e6bd76e40 2024-05-27T07:04:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.899+0000 7efdf7638b80 1 -- 172.21.15.16:0/1745875871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 msgr2=0x562e6befb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.899+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdd400b2e0 tx=0x7efdd4009700 comp rx=0 tx=0).stop 2024-05-27T07:04:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.900+0000 7efdf7638b80 1 -- 172.21.15.16:0/1745875871 shutdown_connections 2024-05-27T07:04:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.900+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bef61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.900+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6befb7f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.900+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1745875871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.900+0000 7efdf7638b80 1 -- 172.21.15.16:0/1745875871 >> 172.21.15.16:0/1745875871 conn(0x562e6bdd6fc0 msgr2=0x562e6bef08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.902+0000 7efdf7638b80 1 -- 172.21.15.16:0/1745875871 shutdown_connections 2024-05-27T07:04:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.902+0000 7efdf7638b80 1 -- 172.21.15.16:0/1745875871 wait complete. 2024-05-27T07:04:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.903+0000 7efdf7638b80 1 -- start start 2024-05-27T07:04:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.904+0000 7efdf7638b80 1 Processor -- start 2024-05-27T07:04:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.905+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6beff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.905+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6beff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6beffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efdf7638b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e6bd19fa0 con 0x562e6befbd30 2024-05-27T07:04:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efdf7638b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e6bdb4760 con 0x562e6bef5db0 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efdf7638b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562e6bf027f0 con 0x562e6bd76e40 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efdf7638b80 10 client.0 _ll_register_callbacks cb 0x562e6bdbdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6beff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6beff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55528/0 (socket says 172.21.15.16:55528) 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efde1e00700 1 -- 172.21.15.16:0/1429636054 learned_addr learned my addr 172.21.15.16:0/1429636054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.906+0000 7efde1400700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6beff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.907+0000 7efde2800700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6beffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.907+0000 7efde1400700 1 -- 172.21.15.16:0/1429636054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 msgr2=0x562e6beff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.907+0000 7efde1400700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6beff460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.907+0000 7efde1400700 1 -- 172.21.15.16:0/1429636054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 msgr2=0x562e6beffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.908+0000 7efde1400700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6beffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.908+0000 7efde1400700 1 -- 172.21.15.16:0/1429636054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6bd638d0 con 0x562e6bef5db0 2024-05-27T07:04:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.908+0000 7efde2800700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6beffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.909+0000 7efde1400700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6beff9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b570 tx=0x7efdd0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a1a06c41d2fc60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.909+0000 7efdc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.910+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0022070 con 0x562e6bef5db0 2024-05-27T07:04:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.910+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7efdd0011db0 con 0x562e6bef5db0 2024-05-27T07:04:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.910+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd4005640 con 0x562e6bef5db0 2024-05-27T07:04:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.911+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdd002e050 con 0x562e6bef5db0 2024-05-27T07:04:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.911+0000 7efdf7638b80 10 client.35708 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.911+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e6bde4060 con 0x562e6bef5db0 2024-05-27T07:04:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.912+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdd4005640 con 0x562e6bef5db0 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.912+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efdd00264d0 con 0x562e6bef5db0 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.912+0000 7efdf7638b80 10 client.35708 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.912+0000 7efdf7638b80 10 client.35708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.913+0000 7efdf7638b80 20 client.35708 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.913+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf0f680 0x562e6bf11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.913+0000 7efdf7638b80 4 client.35708 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.913+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6be25d90 con 0x562e6bf0f680 2024-05-27T07:04:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.913+0000 7efde1e00700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf0f680 0x562e6bf11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.915+0000 7efde1e00700 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf0f680 0x562e6bf11aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7e36db7e8e59897 server_cookie=3e148548f223a514 in_seq=0 out_seq=0 2024-05-27T07:04:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.915+0000 7efdc3e00700 10 client.35708 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:04:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.917+0000 7efdc3e00700 1 -- 172.21.15.16:0/1429636054 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562e6be25d90 con 0x562e6bf0f680 2024-05-27T07:04:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.917+0000 7efdc3e00700 10 client.35708 handle_command_reply: tid=0 2024-05-27T07:04:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.917+0000 7efdf7638b80 1 client.35708 shutdown 2024-05-27T07:04:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.917+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 msgr2=0x562e6beff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.917+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6beff9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b570 tx=0x7efdd0004750 comp rx=0 tx=0).stop 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 shutdown_connections 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf0f680 0x562e6bf11aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6beffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6beff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1429636054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6beff9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdc3e00700 0 client.35708 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.918+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 >> 172.21.15.16:0/1429636054 conn(0x562e6bdd6fc0 msgr2=0x562e6bef0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.919+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 shutdown_connections 2024-05-27T07:04:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.919+0000 7efdf7638b80 1 -- 172.21.15.16:0/1429636054 wait complete. 2024-05-27T07:04:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.919+0000 7efdf7638b80 20 client.35708 trim_cache size 0 max 16384 2024-05-27T07:04:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 Processor -- start 2024-05-27T07:04:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 -- start start 2024-05-27T07:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bd76e40 0x562e6bef5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bef5db0 0x562e6bf0b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b2da0 con 0x562e6befbd30 2024-05-27T07:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b48e0 con 0x562e6bd76e40 2024-05-27T07:04:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.938+0000 7efdf7638b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b3470 con 0x562e6bef5db0 2024-05-27T07:04:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.939+0000 7efde2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.939+0000 7efde2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47902/0 (socket says 172.21.15.16:47902) 2024-05-27T07:04:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.939+0000 7efde2800700 1 -- 172.21.15.16:0/1440261432 learned_addr learned my addr 172.21.15.16:0/1440261432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.939+0000 7efde1400700 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bef5db0 0x562e6bf0b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.940+0000 7efde2800700 1 -- 172.21.15.16:0/1440261432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bef5db0 msgr2=0x562e6bf0b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.940+0000 7efde2800700 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bef5db0 0x562e6bf0b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.940+0000 7efde2800700 1 -- 172.21.15.16:0/1440261432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bd76e40 msgr2=0x562e6bef5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.940+0000 7efde2800700 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bd76e40 0x562e6bef5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.940+0000 7efde2800700 1 -- 172.21.15.16:0/1440261432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6bf00250 con 0x562e6befbd30 2024-05-27T07:04:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.941+0000 7efde2800700 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efddc0a6740 tx=0x7efddc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ba22d3afc8a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.942+0000 7efdcea00700 1 -- 172.21.15.16:0/1440261432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc004d60 con 0x562e6befbd30 2024-05-27T07:04:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.942+0000 7efdcea00700 1 -- 172.21.15.16:0/1440261432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efddc0a9bd0 con 0x562e6befbd30 2024-05-27T07:04:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.942+0000 7efdcea00700 1 -- 172.21.15.16:0/1440261432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0b1e10 con 0x562e6befbd30 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 -- 172.21.15.16:0/1440261432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 msgr2=0x562e6bef9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efddc0a6740 tx=0x7efddc003a00 comp rx=0 tx=0).stop 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 -- 172.21.15.16:0/1440261432 shutdown_connections 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bef9920 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bd76e40 0x562e6bef5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 --2- 172.21.15.16:0/1440261432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bef5db0 0x562e6bf0b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.943+0000 7efdf7638b80 1 -- 172.21.15.16:0/1440261432 >> 172.21.15.16:0/1440261432 conn(0x562e6bdd6fc0 msgr2=0x562e6bef9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.944+0000 7efdf7638b80 1 -- 172.21.15.16:0/1440261432 shutdown_connections 2024-05-27T07:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.944+0000 7efdf7638b80 1 -- 172.21.15.16:0/1440261432 wait complete. 2024-05-27T07:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.945+0000 7efdf7638b80 1 -- start start 2024-05-27T07:04:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.946+0000 7efdf7638b80 1 Processor -- start 2024-05-27T07:04:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.947+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.947+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bf16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.948+0000 7efdf7638b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bf17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.948+0000 7efdf7638b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b4210 con 0x562e6befbd30 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.948+0000 7efdf7638b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b3b40 con 0x562e6bef5db0 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.948+0000 7efdf7638b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b0d40 con 0x562e6bd76e40 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.948+0000 7efdf7638b80 10 client.0 _ll_register_callbacks cb 0x562e6bdbdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.949+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.949+0000 7efde1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55554/0 (socket says 172.21.15.16:55554) 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.949+0000 7efde1e00700 1 -- 172.21.15.16:0/4188156717 learned_addr learned my addr 172.21.15.16:0/4188156717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.949+0000 7efde1400700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bf16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.949+0000 7efde2800700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bf17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1e00700 1 -- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 msgr2=0x562e6bf16c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1e00700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bf16c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1e00700 1 -- 172.21.15.16:0/4188156717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 msgr2=0x562e6bf17180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1e00700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bf17180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1e00700 1 -- 172.21.15.16:0/4188156717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6bf17950 con 0x562e6bd76e40 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde1400700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bf16c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.950+0000 7efde2800700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bf17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.951+0000 7efdc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:04:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.951+0000 7efde1e00700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009990 tx=0x7efdd400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c4a907b557f620f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.952+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4031020 con 0x562e6bd76e40 2024-05-27T07:04:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.952+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efddc0a9bd0 con 0x562e6bd76e40 2024-05-27T07:04:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.952+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7efdd4027040 con 0x562e6bd76e40 2024-05-27T07:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.953+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdd4005680 con 0x562e6bd76e40 2024-05-27T07:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.953+0000 7efdf7638b80 10 client.25648 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.953+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e6bf7a120 con 0x562e6bd76e40 2024-05-27T07:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efddc0a9bd0 con 0x562e6bd76e40 2024-05-27T07:04:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efdd400f460 con 0x562e6bd76e40 2024-05-27T07:04:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 10 client.25648 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 10 client.25648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:04:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 20 client.25648 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf7a400 0x562e6bf7c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 4 client.25648 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.954+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e6be1f570 con 0x562e6bf7a400 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.955+0000 7efde1400700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf7a400 0x562e6bf7c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.956+0000 7efdc3e00700 10 client.25648 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.956+0000 7efde1400700 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf7a400 0x562e6bf7c820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26167550808a1aa server_cookie=dfc12cd607cc2f83 in_seq=0 out_seq=0 2024-05-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.957+0000 7efdc3e00700 1 -- 172.21.15.16:0/4188156717 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e6be1f570 con 0x562e6bf7a400 2024-05-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.957+0000 7efdc3e00700 10 client.25648 handle_command_reply: tid=0 2024-05-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.957+0000 7efdf7638b80 1 client.25648 shutdown 2024-05-27T07:04:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.958+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 msgr2=0x562e6bf08240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.958+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009990 tx=0x7efdd400b900 comp rx=0 tx=0).stop 2024-05-27T07:04:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 shutdown_connections 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x562e6bf7a400 0x562e6bf7c820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e6befbd30 0x562e6bf17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e6bd76e40 0x562e6bf08240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdc3e00700 0 client.25648 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 --2- 172.21.15.16:0/4188156717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6bef5db0 0x562e6bf16c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 >> 172.21.15.16:0/4188156717 conn(0x562e6bdd6fc0 msgr2=0x562e6beff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 shutdown_connections 2024-05-27T07:04:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.959+0000 7efdf7638b80 1 -- 172.21.15.16:0/4188156717 wait complete. 2024-05-27T07:04:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.960+0000 7efdf7638b80 20 client.25648 trim_cache size 0 max 16384 2024-05-27T07:04:15.952 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 msgr2=0x7efdc80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 0x7efdc80b5000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005950 tx=0x7efdd00058e0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 msgr2=0x7efde41a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efddc00ef90 tx=0x7efddc00c450 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 shutdown_connections 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7efdc80b2af0 0x7efdc80b5000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efde41a1900 0x7efde41a1d10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde4076440 0x7efde419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 --2- 172.21.15.16:0/3865917706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4110bf0 0x7efde419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 >> 172.21.15.16:0/3865917706 conn(0x7efde4100030 msgr2=0x7efde4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.961+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 shutdown_connections 2024-05-27T07:04:15.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:15.962+0000 7efde8a00700 1 -- 172.21.15.16:0/3865917706 wait complete. 2024-05-27T07:04:16.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:15 smithi164 ceph-mon[35327]: pgmap v103: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:16.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:15 smithi069 ceph-mon[35276]: pgmap v103: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 83 MiB/s wr, 65 op/s 2024-05-27T07:04:17.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:04:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:17.956 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:17 smithi016 ceph-mon[31415]: pgmap v104: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 127 MiB/s wr, 92 op/s 2024-05-27T07:04:18.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:17 smithi164 ceph-mon[35327]: pgmap v104: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 127 MiB/s wr, 92 op/s 2024-05-27T07:04:18.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:17 smithi069 ceph-mon[35276]: pgmap v104: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 127 MiB/s wr, 92 op/s 2024-05-27T07:04:19.943 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:19 smithi016 ceph-mon[31415]: pgmap v105: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:04:19.943 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:19 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:04:20.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:19 smithi164 ceph-mon[35327]: pgmap v105: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:04:20.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:19 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:04:20.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:19 smithi069 ceph-mon[35276]: pgmap v105: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 87 MiB/s wr, 52 op/s 2024-05-27T07:04:20.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:19 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.439+0000 7f18b4000700 1 -- 172.21.15.16:0/2915029366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac108730 msgr2=0x7f18ac110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.439+0000 7f18b4000700 1 --2- 172.21.15.16:0/2915029366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac108730 0x7f18ac110090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b0d0 tx=0x7f18a400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 -- 172.21.15.16:0/2915029366 shutdown_connections 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 --2- 172.21.15.16:0/2915029366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac108730 0x7f18ac110090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 --2- 172.21.15.16:0/2915029366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac1034b0 0x7f18ac1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 --2- 172.21.15.16:0/2915029366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac107de0 0x7f18ac1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 -- 172.21.15.16:0/2915029366 >> 172.21.15.16:0/2915029366 conn(0x7f18ac0fedb0 msgr2=0x7f18ac1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 -- 172.21.15.16:0/2915029366 shutdown_connections 2024-05-27T07:04:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.440+0000 7f18b4000700 1 -- 172.21.15.16:0/2915029366 wait complete. 2024-05-27T07:04:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.442+0000 7f18b4000700 1 Processor -- start 2024-05-27T07:04:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.442+0000 7f18b4000700 1 -- start start 2024-05-27T07:04:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac1034b0 0x7f18ac1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac108730 0x7f18ac19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac113260 con 0x7f18ac108730 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1130e0 con 0x7f18ac1034b0 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.443+0000 7f18b4000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1133e0 con 0x7f18ac107de0 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.444+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.444+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55566/0 (socket says 172.21.15.16:55566) 2024-05-27T07:04:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.444+0000 7f18b0c00700 1 -- 172.21.15.16:0/461125474 learned_addr learned my addr 172.21.15.16:0/461125474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b1600700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac1034b0 0x7f18ac1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b0c00700 1 -- 172.21.15.16:0/461125474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac1034b0 msgr2=0x7f18ac1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b0c00700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac1034b0 0x7f18ac1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b0c00700 1 -- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac108730 msgr2=0x7f18ac19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b2000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac108730 0x7f18ac19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b0c00700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac108730 0x7f18ac19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.445+0000 7f18b0c00700 1 -- 172.21.15.16:0/461125474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac19b280 con 0x7f18ac107de0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.446+0000 7f18b0c00700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f189800e970 tx=0x7f189800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.447+0000 7f18aa000700 1 -- 172.21.15.16:0/461125474 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800cbd0 con 0x7f18ac107de0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.447+0000 7f18aa000700 1 -- 172.21.15.16:0/461125474 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f189800cd70 con 0x7f18ac107de0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.447+0000 7f18aa000700 1 -- 172.21.15.16:0/461125474 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800cbd0 con 0x7f18ac107de0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.447+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac19b570 con 0x7f18ac107de0 2024-05-27T07:04:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.448+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac1a7770 con 0x7f18ac107de0 2024-05-27T07:04:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.456+0000 7f18aa000700 1 -- 172.21.15.16:0/461125474 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1898005400 con 0x7f18ac107de0 2024-05-27T07:04:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.457+0000 7f18c2c30b80 1 Processor -- start 2024-05-27T07:04:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.458+0000 7f18c2c30b80 1 -- start start 2024-05-27T07:04:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 0x5609fe2c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18aa000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 0x7f18940b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe2d0510 0x5609fe2d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609fe1efe40 con 0x5609fe22e9f0 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609fe1df9e0 con 0x5609fe2d0510 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5609fe1e3a20 con 0x5609fe142e40 2024-05-27T07:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.459+0000 7f18aa000700 1 -- 172.21.15.16:0/461125474 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f18980e2810 con 0x7f18ac107de0 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.460+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.460+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55568/0 (socket says 172.21.15.16:55568) 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.460+0000 7f18b0c00700 1 -- 172.21.15.16:0/3132157880 learned_addr learned my addr 172.21.15.16:0/3132157880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.460+0000 7f18b2000700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe2d0510 0x5609fe2d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.460+0000 7f18b1600700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 0x7f18940b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 -- 172.21.15.16:0/3132157880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe2d0510 msgr2=0x5609fe2d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe2d0510 0x5609fe2d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 -- 172.21.15.16:0/3132157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 msgr2=0x5609fe2c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b2000700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 0x5609fe2c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 0x5609fe2c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 -- 172.21.15.16:0/3132157880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609fe1a8420 con 0x5609fe142e40 2024-05-27T07:04:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b2000700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 0x5609fe2c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.461+0000 7f18b0c00700 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18980a57e0 tx=0x7f18980a8ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4124611e7263d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.462+0000 7f189ea00700 1 -- 172.21.15.16:0/3132157880 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189804e9e0 con 0x5609fe142e40 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.462+0000 7f189ea00700 1 -- 172.21.15.16:0/3132157880 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980cc0b0 con 0x5609fe142e40 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.462+0000 7f189ea00700 1 -- 172.21.15.16:0/3132157880 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980b25c0 con 0x5609fe142e40 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 -- 172.21.15.16:0/3132157880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 msgr2=0x5609fe2c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18980a57e0 tx=0x7f18980a8ed0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 -- 172.21.15.16:0/3132157880 shutdown_connections 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe2d0510 0x5609fe2d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe22e9f0 0x5609fe2c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/3132157880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe142e40 0x5609fe2c1930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.463+0000 7f18c2c30b80 1 -- 172.21.15.16:0/3132157880 >> 172.21.15.16:0/3132157880 conn(0x5609fe1dc390 msgr2=0x5609fe2bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.464+0000 7f18b1600700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 0x7f18940b5180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18a0005fd0 tx=0x7f18a0005f60 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.464+0000 7f18c2c30b80 1 -- 172.21.15.16:0/3132157880 shutdown_connections 2024-05-27T07:04:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.464+0000 7f18c2c30b80 1 -- 172.21.15.16:0/3132157880 wait complete. 2024-05-27T07:04:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.465+0000 7f18c2c30b80 1 -- start start 2024-05-27T07:04:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.466+0000 7f18c2c30b80 1 Processor -- start 2024-05-27T07:04:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.466+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe142e40 0x5609fe2cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.466+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.466+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe2d0510 0x5609fe2c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.467+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609fe0e5fa0 con 0x5609fe142e40 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.467+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609fe180760 con 0x5609fe22e9f0 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.467+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5609fe2d3020 con 0x5609fe2d0510 2024-05-27T07:04:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.467+0000 7f18c2c30b80 10 client.0 _ll_register_callbacks cb 0x5609fe189c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.467+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38582/0 (socket says 172.21.15.16:38582) 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 -- 172.21.15.16:0/2229667088 learned_addr learned my addr 172.21.15.16:0/2229667088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b2000700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe2d0510 0x5609fe2c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 -- 172.21.15.16:0/2229667088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe2d0510 msgr2=0x5609fe2c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe2d0510 0x5609fe2c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 -- 172.21.15.16:0/2229667088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe142e40 msgr2=0x5609fe2cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe142e40 0x5609fe2cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.468+0000 7f18b0c00700 1 -- 172.21.15.16:0/2229667088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609fe12f8d0 con 0x5609fe22e9f0 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.469+0000 7f18b0c00700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18980a96f0 tx=0x7f18980a9270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baff27389ed64e7b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.469+0000 7f1893400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.469+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898091320 con 0x5609fe22e9f0 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.469+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f18980e2410 con 0x5609fe22e9f0 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.469+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18980cc0b0 con 0x5609fe22e9f0 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.470+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18980e2650 con 0x5609fe22e9f0 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.470+0000 7f18c2c30b80 10 client.35723 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.470+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609fe1b0060 con 0x5609fe22e9f0 2024-05-27T07:04:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.470+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980cc0b0 con 0x5609fe22e9f0 2024-05-27T07:04:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.472+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1898050340 con 0x5609fe22e9f0 2024-05-27T07:04:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.472+0000 7f18c2c30b80 10 client.35723 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.472+0000 7f18c2c30b80 10 client.35723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.473+0000 7f18c2c30b80 20 client.35723 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.473+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe2e4320 0x5609fe2e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.473+0000 7f18c2c30b80 4 client.35723 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.473+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609fe1f1d90 con 0x5609fe2e4320 2024-05-27T07:04:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.474+0000 7f18b1600700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe2e4320 0x5609fe2e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.475+0000 7f18b1600700 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe2e4320 0x5609fe2e6740 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3601ef9a9a2e5bc server_cookie=3daa3918ee6d0ec7 in_seq=0 out_seq=0 2024-05-27T07:04:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.475+0000 7f1893400700 10 client.35723 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.477+0000 7f1893400700 1 -- 172.21.15.16:0/2229667088 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609fe1f1d90 con 0x5609fe2e4320 2024-05-27T07:04:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.477+0000 7f1893400700 10 client.35723 handle_command_reply: tid=0 2024-05-27T07:04:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.477+0000 7f18c2c30b80 1 client.35723 shutdown 2024-05-27T07:04:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.477+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 msgr2=0x5609fe2c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.477+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18980a96f0 tx=0x7f18980a9270 comp rx=0 tx=0).stop 2024-05-27T07:04:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 shutdown_connections 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe2e4320 0x5609fe2e6740 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe2d0510 0x5609fe2c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe142e40 0x5609fe2cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2229667088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe22e9f0 0x5609fe2c9070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 >> 172.21.15.16:0/2229667088 conn(0x5609fe1dc390 msgr2=0x5609fe2c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.478+0000 7f1893400700 0 client.35723 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.479+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 shutdown_connections 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.479+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2229667088 wait complete. 2024-05-27T07:04:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.479+0000 7f18c2c30b80 20 client.35723 trim_cache size 0 max 16384 2024-05-27T07:04:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.502+0000 7f18c2c30b80 1 Processor -- start 2024-05-27T07:04:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 -- start start 2024-05-27T07:04:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1ae390 con 0x5609fe2d0510 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1afed0 con 0x5609fe142e40 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.503+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1aea60 con 0x5609fe22e9f0 2024-05-27T07:04:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.504+0000 7f18b2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.505+0000 7f18b2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47952/0 (socket says 172.21.15.16:47952) 2024-05-27T07:04:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.505+0000 7f18b2000700 1 -- 172.21.15.16:0/223860056 learned_addr learned my addr 172.21.15.16:0/223860056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.505+0000 7f18b0c00700 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b0c00700 1 -- 172.21.15.16:0/223860056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 msgr2=0x5609fe2c1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b0c00700 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b0c00700 1 -- 172.21.15.16:0/223860056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 msgr2=0x5609fe2d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b0c00700 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b0c00700 1 -- 172.21.15.16:0/223860056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609fe2c75e0 con 0x5609fe22e9f0 2024-05-27T07:04:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.506+0000 7f18b2000700 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.508+0000 7f18b0c00700 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2d1930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18980a69d0 tx=0x7f189800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180d1cf8299de3a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.509+0000 7f189e000700 1 -- 172.21.15.16:0/223860056 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898091320 con 0x5609fe22e9f0 2024-05-27T07:04:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.509+0000 7f189e000700 1 -- 172.21.15.16:0/223860056 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980b25c0 con 0x5609fe22e9f0 2024-05-27T07:04:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.509+0000 7f189e000700 1 -- 172.21.15.16:0/223860056 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980a9760 con 0x5609fe22e9f0 2024-05-27T07:04:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.510+0000 7f18c2c30b80 1 -- 172.21.15.16:0/223860056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 msgr2=0x5609fe2d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.510+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2d1930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18980a69d0 tx=0x7f189800ef70 comp rx=0 tx=0).stop 2024-05-27T07:04:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.511+0000 7f18c2c30b80 1 -- 172.21.15.16:0/223860056 shutdown_connections 2024-05-27T07:04:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.511+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.511+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2c1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.511+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/223860056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2d1930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.511+0000 7f18c2c30b80 1 -- 172.21.15.16:0/223860056 >> 172.21.15.16:0/223860056 conn(0x5609fe1dc390 msgr2=0x5609fe2c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.512+0000 7f18c2c30b80 1 -- 172.21.15.16:0/223860056 shutdown_connections 2024-05-27T07:04:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.512+0000 7f18c2c30b80 1 -- 172.21.15.16:0/223860056 wait complete. 2024-05-27T07:04:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.512+0000 7f18c2c30b80 1 -- start start 2024-05-27T07:04:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 Processor -- start 2024-05-27T07:04:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2de010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1af800 con 0x5609fe2d0510 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1af130 con 0x5609fe142e40 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.513+0000 7f18c2c30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1ac330 con 0x5609fe22e9f0 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18c2c30b80 10 client.0 _ll_register_callbacks cb 0x5609fe189c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b1600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2de010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47962/0 (socket says 172.21.15.16:47962) 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b1600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2ddad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38610/0 (socket says 172.21.15.16:38610) 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.514+0000 7f18b2000700 1 -- 172.21.15.16:0/2066849022 learned_addr learned my addr 172.21.15.16:0/2066849022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.515+0000 7f18b2000700 1 -- 172.21.15.16:0/2066849022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 msgr2=0x5609fe2de010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.515+0000 7f18b2000700 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2de010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.515+0000 7f18b2000700 1 -- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 msgr2=0x5609fe2ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.515+0000 7f18b2000700 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.515+0000 7f18b2000700 1 -- 172.21.15.16:0/2066849022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609fe34cb80 con 0x5609fe2d0510 2024-05-27T07:04:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.516+0000 7f18b2000700 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005bd0 tx=0x7f18a400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33c361e3f1c576c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.516+0000 7f1893400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.516+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40094a0 con 0x5609fe2d0510 2024-05-27T07:04:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.517+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f18a40079d0 con 0x5609fe2d0510 2024-05-27T07:04:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.517+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18980b25c0 con 0x5609fe2d0510 2024-05-27T07:04:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.517+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18a400be40 con 0x5609fe2d0510 2024-05-27T07:04:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.517+0000 7f18c2c30b80 10 client.15996 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.517+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609fe34ddb0 con 0x5609fe2d0510 2024-05-27T07:04:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.518+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980b25c0 con 0x5609fe2d0510 2024-05-27T07:04:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.518+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f18a4011070 con 0x5609fe2d0510 2024-05-27T07:04:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 20 client.15996 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe34e4a0 0x5609fe3508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 4 client.15996 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609fe1eb570 con 0x5609fe34e4a0 2024-05-27T07:04:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.519+0000 7f18b1600700 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe34e4a0 0x5609fe3508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.521+0000 7f18b1600700 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe34e4a0 0x5609fe3508c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae622511a1596b9f server_cookie=23d6067fe131c769 in_seq=0 out_seq=0 2024-05-27T07:04:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.521+0000 7f1893400700 10 client.15996 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.521+0000 7f1893400700 1 -- 172.21.15.16:0/2066849022 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609fe1eb570 con 0x5609fe34e4a0 2024-05-27T07:04:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.521+0000 7f1893400700 10 client.15996 handle_command_reply: tid=0 2024-05-27T07:04:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.521+0000 7f18c2c30b80 1 client.15996 shutdown 2024-05-27T07:04:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 msgr2=0x5609fe2de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005bd0 tx=0x7f18a400bcd0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 shutdown_connections 2024-05-27T07:04:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5609fe34e4a0 0x5609fe3508c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609fe2d0510 0x5609fe2de550 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609fe142e40 0x5609fe2ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 --2- 172.21.15.16:0/2066849022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609fe22e9f0 0x5609fe2de010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.522+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 >> 172.21.15.16:0/2066849022 conn(0x5609fe1dc390 msgr2=0x5609fe2bce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.523+0000 7f1893400700 0 client.15996 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.523+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 shutdown_connections 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.523+0000 7f18c2c30b80 1 -- 172.21.15.16:0/2066849022 wait complete. 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.523+0000 7f18c2c30b80 20 client.15996 trim_cache size 0 max 16384 2024-05-27T07:04:20.516 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 msgr2=0x7f18940b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 0x7f18940b5180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18a0005fd0 tx=0x7f18a0005f60 comp rx=0 tx=0).stop 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 msgr2=0x7f18ac1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f189800e970 tx=0x7f189800ed30 comp rx=0 tx=0).stop 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 shutdown_connections 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f18940b2c70 0x7f18940b5180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ac108730 0x7f18ac19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac1034b0 0x7f18ac1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 --2- 172.21.15.16:0/461125474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18ac107de0 0x7f18ac1a0f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 >> 172.21.15.16:0/461125474 conn(0x7f18ac0fedb0 msgr2=0x7f18ac109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.524+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 shutdown_connections 2024-05-27T07:04:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:20.525+0000 7f18b4000700 1 -- 172.21.15.16:0/461125474 wait complete. 2024-05-27T07:04:22.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:21 smithi164 ceph-mon[35327]: pgmap v106: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:22.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:21 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:04:22.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:21 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:04:22.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:21 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:04:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:21 smithi016 ceph-mon[31415]: pgmap v106: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:21 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:04:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:21 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:04:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:21 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:04:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:21 smithi069 ceph-mon[35276]: pgmap v106: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:21 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:04:22.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:21 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:04:22.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:21 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:04:23.335 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:04:23.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:04:23.689 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:23 smithi016 ceph-mon[31415]: pgmap v107: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:23 smithi164 ceph-mon[35327]: pgmap v107: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:24.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:23 smithi069 ceph-mon[35276]: pgmap v107: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:25.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:25 smithi016 ceph-mon[31415]: pgmap v108: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:26.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:25 smithi164 ceph-mon[35327]: pgmap v108: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:26.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:25 smithi069 ceph-mon[35276]: pgmap v108: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 86 MiB/s wr, 52 op/s 2024-05-27T07:04:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 -- 172.21.15.16:0/3808476049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 msgr2=0x7ff18c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 --2- 172.21.15.16:0/3808476049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c109300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff17800fb90 tx=0x7ff17800fea0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 -- 172.21.15.16:0/3808476049 shutdown_connections 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 --2- 172.21.15.16:0/3808476049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 0x7ff18c1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 --2- 172.21.15.16:0/3808476049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 --2- 172.21.15.16:0/3808476049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c109300 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.621+0000 7ff192a00700 1 -- 172.21.15.16:0/3808476049 >> 172.21.15.16:0/3808476049 conn(0x7ff18c0ffe70 msgr2=0x7ff18c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.622+0000 7ff192a00700 1 -- 172.21.15.16:0/3808476049 shutdown_connections 2024-05-27T07:04:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.622+0000 7ff192a00700 1 -- 172.21.15.16:0/3808476049 wait complete. 2024-05-27T07:04:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.623+0000 7ff192a00700 1 Processor -- start 2024-05-27T07:04:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.623+0000 7ff192a00700 1 -- start start 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 0x7ff18c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c11aa00 con 0x7ff18c108ef0 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c11a880 con 0x7ff18c1045c0 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff192a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c11ab80 con 0x7ff18c109840 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff18be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff18be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:47112/0 (socket says 172.21.15.16:47112) 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff18be00700 1 -- 172.21.15.16:0/1594790775 learned_addr learned my addr 172.21.15.16:0/1594790775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.624+0000 7ff18b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff190a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 0x7ff18c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff18b400700 1 -- 172.21.15.16:0/1594790775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 msgr2=0x7ff18c1a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff18b400700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 0x7ff18c1a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff18b400700 1 -- 172.21.15.16:0/1594790775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 msgr2=0x7ff18c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff18b400700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.625+0000 7ff18b400700 1 -- 172.21.15.16:0/1594790775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c1a63a0 con 0x7ff18c108ef0 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.626+0000 7ff18b400700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c1a1a70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff17800fa80 tx=0x7ff17800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.626+0000 7ff188c00700 1 -- 172.21.15.16:0/1594790775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17801c070 con 0x7ff18c108ef0 2024-05-27T07:04:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.626+0000 7ff188c00700 1 -- 172.21.15.16:0/1594790775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1780050e0 con 0x7ff18c108ef0 2024-05-27T07:04:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.626+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18c1a6630 con 0x7ff18c108ef0 2024-05-27T07:04:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.627+0000 7ff188c00700 1 -- 172.21.15.16:0/1594790775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178003970 con 0x7ff18c108ef0 2024-05-27T07:04:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.627+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18c1a6b10 con 0x7ff18c108ef0 2024-05-27T07:04:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.634+0000 7ff1a160fb80 1 Processor -- start 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.635+0000 7ff188c00700 1 -- 172.21.15.16:0/1594790775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff17800c040 con 0x7ff18c108ef0 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.635+0000 7ff1a160fb80 1 -- start start 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.635+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 0x5600080e7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560008014ec0 con 0x560007f67ec0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560008004a60 con 0x5600080ecdb0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560008008aa0 con 0x5600080e6e30 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff190a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff190a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.636+0000 7ff190a00700 1 -- 172.21.15.16:0/3470699025 learned_addr learned my addr 172.21.15.16:0/3470699025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.637+0000 7ff18be00700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.637+0000 7ff18b400700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 0x5600080e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.637+0000 7ff188c00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 0x7ff1700b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff188c00700 1 -- 172.21.15.16:0/1594790775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff1780e2810 con 0x7ff18c108ef0 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 -- 172.21.15.16:0/3470699025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 msgr2=0x5600080e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 -- 172.21.15.16:0/3470699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 msgr2=0x5600080ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 -- 172.21.15.16:0/3470699025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560007fcd4a0 con 0x5600080e6e30 2024-05-27T07:04:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff190a00700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.638+0000 7ff18b400700 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 0x5600080e7240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1780dfb70 tx=0x7ff17800fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac888d3dbfc6668 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.639+0000 7ff190a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 0x7ff1700b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.639+0000 7ff181600700 1 -- 172.21.15.16:0/3470699025 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178025070 con 0x5600080e6e30 2024-05-27T07:04:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.639+0000 7ff181600700 1 -- 172.21.15.16:0/3470699025 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1780a92a0 con 0x5600080e6e30 2024-05-27T07:04:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.639+0000 7ff181600700 1 -- 172.21.15.16:0/3470699025 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1780a9790 con 0x5600080e6e30 2024-05-27T07:04:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.640+0000 7ff1a160fb80 1 -- 172.21.15.16:0/3470699025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 msgr2=0x5600080e7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.640+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 0x5600080e7240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1780dfb70 tx=0x7ff17800fea0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.641+0000 7ff190a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 0x7ff1700b50d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1840079e0 tx=0x7ff184009920 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 -- 172.21.15.16:0/3470699025 shutdown_connections 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080e6e30 0x5600080e7240 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560007f67ec0 0x5600080ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/3470699025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 -- 172.21.15.16:0/3470699025 >> 172.21.15.16:0/3470699025 conn(0x560007fc8040 msgr2=0x5600080e1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 -- 172.21.15.16:0/3470699025 shutdown_connections 2024-05-27T07:04:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.642+0000 7ff1a160fb80 1 -- 172.21.15.16:0/3470699025 wait complete. 2024-05-27T07:04:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.644+0000 7ff1a160fb80 1 -- start start 2024-05-27T07:04:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.644+0000 7ff1a160fb80 1 Processor -- start 2024-05-27T07:04:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560007f0b020 con 0x5600080e6e30 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560007fa57e0 con 0x560007f67ec0 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.645+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560008008aa0 con 0x5600080ecdb0 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff1a160fb80 10 client.0 _ll_register_callbacks cb 0x560007faecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55512/0 (socket says 172.21.15.16:55512) 2024-05-27T07:04:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 -- 172.21.15.16:0/4208743285 learned_addr learned my addr 172.21.15.16:0/4208743285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 -- 172.21.15.16:0/4208743285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 msgr2=0x5600080f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 -- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 msgr2=0x5600080f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.646+0000 7ff18b400700 1 -- 172.21.15.16:0/4208743285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560007f54950 con 0x5600080e6e30 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.647+0000 7ff18b400700 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff1780a5810 tx=0x7ff1780a9ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa08281b52e42467 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.647+0000 7ff176000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.647+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178025070 con 0x5600080e6e30 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.647+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff1780b22e0 con 0x5600080e6e30 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.648+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1780a92a0 con 0x5600080e6e30 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.648+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1780a8f00 con 0x5600080e6e30 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.648+0000 7ff1a160fb80 10 client.16008 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.648+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560007fd50e0 con 0x5600080e6e30 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.648+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1780a92a0 con 0x5600080e6e30 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff1780e23b0 con 0x5600080e6e30 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 10 client.16008 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 20 client.16008 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5600081001b0 0x5600081025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 4 client.16008 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.649+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560008016e10 con 0x5600081001b0 2024-05-27T07:04:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.650+0000 7ff18be00700 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5600081001b0 0x5600081025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.651+0000 7ff18be00700 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5600081001b0 0x5600081025d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff41a5d7e0db5067 server_cookie=e505477d671f5e2f in_seq=0 out_seq=0 2024-05-27T07:04:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.651+0000 7ff176000700 10 client.16008 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.654+0000 7ff176000700 1 -- 172.21.15.16:0/4208743285 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560008016e10 con 0x5600081001b0 2024-05-27T07:04:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.654+0000 7ff176000700 10 client.16008 handle_command_reply: tid=0 2024-05-27T07:04:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.654+0000 7ff1a160fb80 1 client.16008 shutdown 2024-05-27T07:04:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.654+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 msgr2=0x5600080f0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.654+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff1780a5810 tx=0x7ff1780a9ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 shutdown_connections 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5600081001b0 0x5600081025d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/4208743285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f0af0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 >> 172.21.15.16:0/4208743285 conn(0x560007fc8040 msgr2=0x5600080e1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff176000700 0 client.16008 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 shutdown_connections 2024-05-27T07:04:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 1 -- 172.21.15.16:0/4208743285 wait complete. 2024-05-27T07:04:26.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.655+0000 7ff1a160fb80 20 client.16008 trim_cache size 0 max 16384 2024-05-27T07:04:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 Processor -- start 2024-05-27T07:04:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 -- start start 2024-05-27T07:04:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b6b30 con 0x5600080e6e30 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b8670 con 0x560007f67ec0 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.688+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b7200 con 0x5600080ecdb0 2024-05-27T07:04:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.689+0000 7ff18b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.689+0000 7ff18b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55520/0 (socket says 172.21.15.16:55520) 2024-05-27T07:04:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.689+0000 7ff18b400700 1 -- 172.21.15.16:0/1881595689 learned_addr learned my addr 172.21.15.16:0/1881595689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.689+0000 7ff190a00700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.689+0000 7ff18be00700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 -- 172.21.15.16:0/1881595689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 msgr2=0x5600080fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 -- 172.21.15.16:0/1881595689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 msgr2=0x5600080e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 -- 172.21.15.16:0/1881595689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600080e27a0 con 0x5600080e6e30 2024-05-27T07:04:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18be00700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.690+0000 7ff18b400700 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff1780a9720 tx=0x7ff1780a97e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0de02c4db9ab09f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.691+0000 7ff180c00700 1 -- 172.21.15.16:0/1881595689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178025070 con 0x5600080e6e30 2024-05-27T07:04:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.691+0000 7ff180c00700 1 -- 172.21.15.16:0/1881595689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1780a9e90 con 0x5600080e6e30 2024-05-27T07:04:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.691+0000 7ff180c00700 1 -- 172.21.15.16:0/1881595689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17800f6d0 con 0x5600080e6e30 2024-05-27T07:04:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.692+0000 7ff1a160fb80 1 -- 172.21.15.16:0/1881595689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 msgr2=0x5600080eabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.692+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff1780a9720 tx=0x7ff1780a97e0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 -- 172.21.15.16:0/1881595689 shutdown_connections 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600080ecdb0 0x5600080fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560007f67ec0 0x5600080e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/1881595689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080eabf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 -- 172.21.15.16:0/1881595689 >> 172.21.15.16:0/1881595689 conn(0x560007fc8040 msgr2=0x5600080eeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 -- 172.21.15.16:0/1881595689 shutdown_connections 2024-05-27T07:04:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.693+0000 7ff1a160fb80 1 -- 172.21.15.16:0/1881595689 wait complete. 2024-05-27T07:04:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.694+0000 7ff1a160fb80 1 -- start start 2024-05-27T07:04:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.694+0000 7ff1a160fb80 1 Processor -- start 2024-05-27T07:04:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 0x5600080f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b7fa0 con 0x5600080e6e30 2024-05-27T07:04:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b78d0 con 0x5600080ecdb0 2024-05-27T07:04:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c1b4ad0 con 0x560007f67ec0 2024-05-27T07:04:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.695+0000 7ff1a160fb80 10 client.0 _ll_register_callbacks cb 0x560007faecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.696+0000 7ff18be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 0x5600080f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.696+0000 7ff190a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.696+0000 7ff190a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:47136/0 (socket says 172.21.15.16:47136) 2024-05-27T07:04:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.696+0000 7ff190a00700 1 -- 172.21.15.16:0/2730691505 learned_addr learned my addr 172.21.15.16:0/2730691505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:26.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18be00700 1 -- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 msgr2=0x5600080f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18be00700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18b400700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18be00700 1 -- 172.21.15.16:0/2730691505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 msgr2=0x5600080f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:26.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18be00700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18be00700 1 -- 172.21.15.16:0/2730691505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600080f5cd0 con 0x560007f67ec0 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.698+0000 7ff18b400700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.699+0000 7ff18be00700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 0x5600080f8710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5600080f1700 tx=0x7ff17c016430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79d3babe900036f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.699+0000 7ff176000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.700+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c016d80 con 0x560007f67ec0 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.700+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff17c01f360 con 0x560007f67ec0 2024-05-27T07:04:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.700+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1780a9e90 con 0x560007f67ec0 2024-05-27T07:04:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.701+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff17c01f560 con 0x560007f67ec0 2024-05-27T07:04:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.701+0000 7ff1a160fb80 10 client.25681 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.701+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56000816b2d0 con 0x560007f67ec0 2024-05-27T07:04:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1780a9e90 con 0x560007f67ec0 2024-05-27T07:04:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff17c01e9a0 con 0x560007f67ec0 2024-05-27T07:04:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 10 client.25681 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 20 client.25681 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56000816b9e0 0x56000816de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 4 client.25681 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:04:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.702+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600080105f0 con 0x56000816b9e0 2024-05-27T07:04:26.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.704+0000 7ff18b400700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56000816b9e0 0x56000816de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:26.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.705+0000 7ff18b400700 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56000816b9e0 0x56000816de00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d582ba719f8e3c server_cookie=68db640e139fc9d9 in_seq=0 out_seq=0 2024-05-27T07:04:26.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.705+0000 7ff176000700 10 client.25681 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.709+0000 7ff176000700 1 -- 172.21.15.16:0/2730691505 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600080105f0 con 0x56000816b9e0 2024-05-27T07:04:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.709+0000 7ff176000700 10 client.25681 handle_command_reply: tid=0 2024-05-27T07:04:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.709+0000 7ff1a160fb80 1 client.25681 shutdown 2024-05-27T07:04:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 msgr2=0x5600080f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 0x5600080f8710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5600080f1700 tx=0x7ff17c016430 comp rx=0 tx=0).stop 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 shutdown_connections 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56000816b9e0 0x56000816de00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600080ecdb0 0x5600080f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560007f67ec0 0x5600080f8710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 --2- 172.21.15.16:0/2730691505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600080e6e30 0x5600080f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 >> 172.21.15.16:0/2730691505 conn(0x560007fc8040 msgr2=0x5600080ea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff176000700 0 client.25681 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.710+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 shutdown_connections 2024-05-27T07:04:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.711+0000 7ff1a160fb80 1 -- 172.21.15.16:0/2730691505 wait complete. 2024-05-27T07:04:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.711+0000 7ff1a160fb80 20 client.25681 trim_cache size 0 max 16384 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "297bc53c-294b-46c3-bda5-41d42f9cf8c7", 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:04:26.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 msgr2=0x7ff1700b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 0x7ff1700b50d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1840079e0 tx=0x7ff184009920 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 msgr2=0x7ff18c1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c1a1a70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff17800fa80 tx=0x7ff17800ef20 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 shutdown_connections 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1700b2bc0 0x7ff1700b50d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff18c109840 0x7ff18c1a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff18c1045c0 0x7ff18c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 --2- 172.21.15.16:0/1594790775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff18c108ef0 0x7ff18c1a1a70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.712+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 >> 172.21.15.16:0/1594790775 conn(0x7ff18c0ffe70 msgr2=0x7ff18c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.713+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 shutdown_connections 2024-05-27T07:04:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:26.713+0000 7ff192a00700 1 -- 172.21.15.16:0/1594790775 wait complete. 2024-05-27T07:04:28.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:27 smithi164 ceph-mon[35327]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:28.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:27 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:04:28.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:27 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:28.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:27 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:04:28.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:27 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:28.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:27 smithi016 ceph-mon[31415]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:27 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:04:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:27 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:27 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:04:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:27 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:27 smithi069 ceph-mon[35276]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 129 MiB/s wr, 78 op/s 2024-05-27T07:04:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:27 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:04:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:27 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:27 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:04:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:27 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:04:30.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:29 smithi164 ceph-mon[35327]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:30.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:29 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:04:30.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:29 smithi016 ceph-mon[31415]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:30.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:29 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:04:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:29 smithi069 ceph-mon[35276]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-05-27T07:04:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:29 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:04:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:31 smithi164 ceph-mon[35327]: pgmap v111: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:32.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:31 smithi016 ceph-mon[31415]: pgmap v111: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:32.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:31 smithi069 ceph-mon[35276]: pgmap v111: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:33.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:04:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:33 smithi164 ceph-mon[35327]: pgmap v112: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:34.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:33 smithi016 ceph-mon[31415]: pgmap v112: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:34.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:33 smithi069 ceph-mon[35276]: pgmap v112: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:35 smithi164 ceph-mon[35327]: pgmap v113: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:35 smithi016 ceph-mon[31415]: pgmap v113: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:36.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:35 smithi069 ceph-mon[35276]: pgmap v113: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 78 MiB/s wr, 47 op/s 2024-05-27T07:04:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 -- 172.21.15.16:0/2333647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa81043f0 msgr2=0x7f6fa8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 --2- 172.21.15.16:0/2333647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa81043f0 0x7f6fa8104800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6f940099c0 tx=0x7f6f94009cd0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 -- 172.21.15.16:0/2333647718 shutdown_connections 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 --2- 172.21.15.16:0/2333647718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa8109670 0x7f6fa8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 --2- 172.21.15.16:0/2333647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa81043f0 0x7f6fa8104800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 --2- 172.21.15.16:0/2333647718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8108d20 0x7f6fa8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 -- 172.21.15.16:0/2333647718 >> 172.21.15.16:0/2333647718 conn(0x7f6fa8079d00 msgr2=0x7f6fa807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 -- 172.21.15.16:0/2333647718 shutdown_connections 2024-05-27T07:04:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.135+0000 7f6fa7200700 1 -- 172.21.15.16:0/2333647718 wait complete. 2024-05-27T07:04:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.137+0000 7f6fa7200700 1 Processor -- start 2024-05-27T07:04:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.137+0000 7f6fa7200700 1 -- start start 2024-05-27T07:04:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa81043f0 0x7f6fa81a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8109670 0x7f6fa81a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa811a880 con 0x7f6fa8108d20 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa811a700 con 0x7f6fa81043f0 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa7200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa811aa00 con 0x7f6fa8109670 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.138+0000 7f6fa5400700 1 -- 172.21.15.16:0/1202283300 learned_addr learned my addr 172.21.15.16:0/1202283300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.139+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa81043f0 0x7f6fa81a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.139+0000 7f6fa6800700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8109670 0x7f6fa81a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.140+0000 7f6fa5400700 1 -- 172.21.15.16:0/1202283300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8109670 msgr2=0x7f6fa81a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.140+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8109670 0x7f6fa81a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.140+0000 7f6fa5400700 1 -- 172.21.15.16:0/1202283300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa81043f0 msgr2=0x7f6fa81a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.140+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa81043f0 0x7f6fa81a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.140+0000 7f6fa5400700 1 -- 172.21.15.16:0/1202283300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa81a6050 con 0x7f6fa8108d20 2024-05-27T07:04:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.141+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c9e0 tx=0x7f6f9800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.142+0000 7f6f9ea00700 1 -- 172.21.15.16:0/1202283300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98007a20 con 0x7f6fa8108d20 2024-05-27T07:04:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.142+0000 7f6f9ea00700 1 -- 172.21.15.16:0/1202283300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f98007bc0 con 0x7f6fa8108d20 2024-05-27T07:04:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.142+0000 7f6f9ea00700 1 -- 172.21.15.16:0/1202283300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98018400 con 0x7f6fa8108d20 2024-05-27T07:04:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.142+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa81a6340 con 0x7f6fa8108d20 2024-05-27T07:04:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.143+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa81a6880 con 0x7f6fa8108d20 2024-05-27T07:04:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.149+0000 7f6f9ea00700 1 -- 172.21.15.16:0/1202283300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6f9800f450 con 0x7f6fa8108d20 2024-05-27T07:04:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.152+0000 7f6f9ea00700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 0x7f6f8c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.153+0000 7f6f9ea00700 1 -- 172.21.15.16:0/1202283300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6f980e2730 con 0x7f6fa8108d20 2024-05-27T07:04:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.153+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 0x7f6f8c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.154+0000 7f6fbb838b80 1 Processor -- start 2024-05-27T07:04:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.155+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 0x7f6f8c0b5090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f940095f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.155+0000 7f6fbb838b80 1 -- start start 2024-05-27T07:04:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.155+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db97b820 0x5654db988020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5654db8a9e40 con 0x5654db7fce40 2024-05-27T07:04:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5654db8999e0 con 0x5654db97b410 2024-05-27T07:04:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654db89da20 con 0x5654db97b820 2024-05-27T07:04:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2024-05-27T07:04:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.156+0000 7f6fa5e00700 1 -- 172.21.15.16:0/471319084 learned_addr learned my addr 172.21.15.16:0/471319084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa5400700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa6800700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db97b820 0x5654db988020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa5400700 1 -- 172.21.15.16:0/471319084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db97b820 msgr2=0x5654db988020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa5400700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db97b820 0x5654db988020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa5400700 1 -- 172.21.15.16:0/471319084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 msgr2=0x5654db97aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.157+0000 7f6fa5400700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.158+0000 7f6fa5400700 1 -- 172.21.15.16:0/471319084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654db862420 con 0x5654db97b410 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.158+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.158+0000 7f6fa5400700 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f980dfaf0 tx=0x7f6f980a9dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f54d271f6369356 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.159+0000 7f6f93400700 1 -- 172.21.15.16:0/471319084 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980b24e0 con 0x5654db97b410 2024-05-27T07:04:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.159+0000 7f6f93400700 1 -- 172.21.15.16:0/471319084 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f980a5910 con 0x5654db97b410 2024-05-27T07:04:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.159+0000 7f6f93400700 1 -- 172.21.15.16:0/471319084 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a98e0 con 0x5654db97b410 2024-05-27T07:04:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.160+0000 7f6fbb838b80 1 -- 172.21.15.16:0/471319084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 msgr2=0x5654db985ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.160+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f980dfaf0 tx=0x7f6f980a9dd0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 -- 172.21.15.16:0/471319084 shutdown_connections 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db97b820 0x5654db988020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db7fce40 0x5654db97aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/471319084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 -- 172.21.15.16:0/471319084 >> 172.21.15.16:0/471319084 conn(0x5654db85cfc0 msgr2=0x5654db976d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 -- 172.21.15.16:0/471319084 shutdown_connections 2024-05-27T07:04:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.161+0000 7f6fbb838b80 1 -- 172.21.15.16:0/471319084 wait complete. 2024-05-27T07:04:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.162+0000 7f6fbb838b80 1 -- start start 2024-05-27T07:04:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.162+0000 7f6fbb838b80 1 Processor -- start 2024-05-27T07:04:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db985600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db986080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5654db79ffa0 con 0x5654db97b820 2024-05-27T07:04:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5654db83a760 con 0x5654db97b410 2024-05-27T07:04:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.163+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654db988a30 con 0x5654db7fce40 2024-05-27T07:04:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fbb838b80 10 client.0 _ll_register_callbacks cb 0x5654db843c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fa5400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db985600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fa5400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37604/0 (socket says 172.21.15.16:37604) 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fa5400700 1 -- 172.21.15.16:0/1645906004 learned_addr learned my addr 172.21.15.16:0/1645906004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.164+0000 7f6fa6800700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db986080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5400700 1 -- 172.21.15.16:0/1645906004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 msgr2=0x5654db985600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db985600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5400700 1 -- 172.21.15.16:0/1645906004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 msgr2=0x5654db986080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db986080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5400700 1 -- 172.21.15.16:0/1645906004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654db7e98d0 con 0x5654db97b410 2024-05-27T07:04:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa6800700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db986080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.165+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db985600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.166+0000 7f6fa5400700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a68f0 tx=0x7f6f98000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6912db9185d0a095 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.166+0000 7f6f87e00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.166+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980b24e0 con 0x5654db97b410 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.166+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6f980a9af0 con 0x5654db97b410 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.166+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f980a5910 con 0x5654db97b410 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.167+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f980b2050 con 0x5654db97b410 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.167+0000 7f6fbb838b80 10 client.35750 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.167+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654db86a060 con 0x5654db97b410 2024-05-27T07:04:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.167+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f980a5910 con 0x5654db97b410 2024-05-27T07:04:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6f980bead0 con 0x5654db97b410 2024-05-27T07:04:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 10 client.35750 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 10 client.35750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 20 client.35750 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654db995880 0x5654db997ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 4 client.35750 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.168+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654db8abd90 con 0x5654db995880 2024-05-27T07:04:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.169+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654db995880 0x5654db997ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.170+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654db995880 0x5654db997ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97f2a98b19166b1 server_cookie=d0396a4491fd2b60 in_seq=0 out_seq=0 2024-05-27T07:04:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.170+0000 7f6f87e00700 10 client.35750 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6f87e00700 1 -- 172.21.15.16:0/1645906004 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654db8abd90 con 0x5654db995880 2024-05-27T07:04:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6f87e00700 10 client.35750 handle_command_reply: tid=0 2024-05-27T07:04:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 client.35750 shutdown 2024-05-27T07:04:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 msgr2=0x5654db985b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a68f0 tx=0x7f6f98000bc0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 shutdown_connections 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654db995880 0x5654db997ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db986080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db985600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/1645906004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db985b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.172+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 >> 172.21.15.16:0/1645906004 conn(0x5654db85cfc0 msgr2=0x5654db975ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.173+0000 7f6f87e00700 0 client.35750 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.173+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 shutdown_connections 2024-05-27T07:04:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.173+0000 7f6fbb838b80 1 -- 172.21.15.16:0/1645906004 wait complete. 2024-05-27T07:04:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.173+0000 7f6fbb838b80 20 client.35750 trim_cache size 0 max 16384 2024-05-27T07:04:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.210+0000 7f6fbb838b80 1 Processor -- start 2024-05-27T07:04:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.210+0000 7f6fbb838b80 1 -- start start 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db980e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 0x5654db9913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 0x5654db99acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b68f0 con 0x5654db97b410 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b8430 con 0x5654db97b820 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b6fc0 con 0x5654db7fce40 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.211+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db980e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.212+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db980e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36600/0 (socket says 172.21.15.16:36600) 2024-05-27T07:04:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.212+0000 7f6fa5e00700 1 -- 172.21.15.16:0/4174598698 learned_addr learned my addr 172.21.15.16:0/4174598698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.212+0000 7f6fa6800700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 0x5654db99acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.212+0000 7f6fa5400700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 0x5654db9913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 -- 172.21.15.16:0/4174598698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 msgr2=0x5654db980e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db980e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 -- 172.21.15.16:0/4174598698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 msgr2=0x5654db99acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 0x5654db99acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 -- 172.21.15.16:0/4174598698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654db9771f0 con 0x5654db97b410 2024-05-27T07:04:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa6800700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 0x5654db99acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.213+0000 7f6fa5400700 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 0x5654db9913d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a9380 tx=0x7f6f980dfba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9231541e80504dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.214+0000 7f6f92a00700 1 -- 172.21.15.16:0/4174598698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a9d80 con 0x5654db97b410 2024-05-27T07:04:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.214+0000 7f6f92a00700 1 -- 172.21.15.16:0/4174598698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f980a9af0 con 0x5654db97b410 2024-05-27T07:04:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.214+0000 7f6f92a00700 1 -- 172.21.15.16:0/4174598698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a5aa0 con 0x5654db97b410 2024-05-27T07:04:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.215+0000 7f6fbb838b80 1 -- 172.21.15.16:0/4174598698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 msgr2=0x5654db9913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.215+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 0x5654db9913d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a9380 tx=0x7f6f980dfba0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.216+0000 7f6fbb838b80 1 -- 172.21.15.16:0/4174598698 shutdown_connections 2024-05-27T07:04:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.216+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b820 0x5654db99acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.216+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db980e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.216+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/4174598698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b410 0x5654db9913d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.216+0000 7f6fbb838b80 1 -- 172.21.15.16:0/4174598698 >> 172.21.15.16:0/4174598698 conn(0x5654db85cfc0 msgr2=0x5654db990cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.218+0000 7f6fbb838b80 1 -- 172.21.15.16:0/4174598698 shutdown_connections 2024-05-27T07:04:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.218+0000 7f6fbb838b80 1 -- 172.21.15.16:0/4174598698 wait complete. 2024-05-27T07:04:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.218+0000 7f6fbb838b80 1 -- start start 2024-05-27T07:04:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.219+0000 7f6fbb838b80 1 Processor -- start 2024-05-27T07:04:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.220+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db99b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.220+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db9f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.220+0000 7f6fbb838b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db9f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.222+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b7d60 con 0x5654db97b820 2024-05-27T07:04:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.222+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b7690 con 0x5654db97b410 2024-05-27T07:04:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.222+0000 7f6fbb838b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81b4890 con 0x5654db7fce40 2024-05-27T07:04:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.222+0000 7f6fbb838b80 10 client.0 _ll_register_callbacks cb 0x5654db843c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.223+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db99b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.223+0000 7f6fa5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db99b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36614/0 (socket says 172.21.15.16:36614) 2024-05-27T07:04:37.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.223+0000 7f6fa5e00700 1 -- 172.21.15.16:0/2084004564 learned_addr learned my addr 172.21.15.16:0/2084004564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:37.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.223+0000 7f6fa6800700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db9f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.224+0000 7f6fa6800700 1 -- 172.21.15.16:0/2084004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 msgr2=0x5654db99b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.224+0000 7f6fa6800700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db99b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.224+0000 7f6fa6800700 1 -- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 msgr2=0x5654db9f8a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.224+0000 7f6fa6800700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db9f8a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.224+0000 7f6fa6800700 1 -- 172.21.15.16:0/2084004564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654db9f95e0 con 0x5654db97b820 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.225+0000 7f6f87e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:04:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.225+0000 7f6fa6800700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db9f8fa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0007af0 tx=0x7f6fa0007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7736b27173216bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004db0 con 0x5654db97b820 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6fa0004280 con 0x5654db97b820 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f980a9af0 con 0x5654db97b820 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0029050 con 0x5654db97b820 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6fbb838b80 10 client.16029 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.226+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654dba110f0 con 0x5654db97b820 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.228+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f980a9af0 con 0x5654db97b820 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.228+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6fa0005720 con 0x5654db97b820 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.228+0000 7f6fbb838b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.228+0000 7f6fbb838b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.228+0000 7f6fbb838b80 20 client.16029 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.229+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654dba116f0 0x5654dba13b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.229+0000 7f6fbb838b80 4 client.16029 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:04:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.229+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654db8a5570 con 0x5654dba116f0 2024-05-27T07:04:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.229+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654dba116f0 0x5654dba13b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.230+0000 7f6fa5e00700 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654dba116f0 0x5654dba13b10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b7886719dd301c server_cookie=7bf4d5461dea8146 in_seq=0 out_seq=0 2024-05-27T07:04:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.230+0000 7f6f87e00700 10 client.16029 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.231+0000 7f6f87e00700 1 -- 172.21.15.16:0/2084004564 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654db8a5570 con 0x5654dba116f0 2024-05-27T07:04:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.231+0000 7f6f87e00700 10 client.16029 handle_command_reply: tid=0 2024-05-27T07:04:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.231+0000 7f6fbb838b80 1 client.16029 shutdown 2024-05-27T07:04:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.231+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 msgr2=0x5654db9f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.231+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db9f8fa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0007af0 tx=0x7f6fa0007b20 comp rx=0 tx=0).stop 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 shutdown_connections 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654dba116f0 0x5654dba13b10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654db97b820 0x5654db9f8fa0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654db7fce40 0x5654db99b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 --2- 172.21.15.16:0/2084004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654db97b410 0x5654db9f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 >> 172.21.15.16:0/2084004564 conn(0x5654db85cfc0 msgr2=0x5654db9853e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6f87e00700 0 client.16029 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.232+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 shutdown_connections 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.233+0000 7f6fbb838b80 1 -- 172.21.15.16:0/2084004564 wait complete. 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.233+0000 7f6fbb838b80 20 client.16029 trim_cache size 0 max 16384 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:04:37.225 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:04:37.226 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.234+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 msgr2=0x7f6f8c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.234+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 0x7f6f8c0b5090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f940095f0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 msgr2=0x7f6fa81a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c9e0 tx=0x7f6f9800ccf0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 shutdown_connections 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6f8c0b2b80 0x7f6f8c0b5090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa8109670 0x7f6fa81a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa81043f0 0x7f6fa81a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 --2- 172.21.15.16:0/1202283300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa8108d20 0x7f6fa81a17d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.235+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 >> 172.21.15.16:0/1202283300 conn(0x7f6fa8079d00 msgr2=0x7f6fa8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.236+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 shutdown_connections 2024-05-27T07:04:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:37.236+0000 7f6fa7200700 1 -- 172.21.15.16:0/1202283300 wait complete. 2024-05-27T07:04:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:37 smithi164 ceph-mon[35327]: pgmap v114: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 115 MiB/s wr, 70 op/s 2024-05-27T07:04:38.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:37 smithi016 ceph-mon[31415]: pgmap v114: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 115 MiB/s wr, 70 op/s 2024-05-27T07:04:38.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:37 smithi069 ceph-mon[35276]: pgmap v114: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 115 MiB/s wr, 70 op/s 2024-05-27T07:04:38.393 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:04:38.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:04:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:39.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:38 smithi164 ceph-mon[35327]: pgmap v115: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 73 MiB/s wr, 45 op/s 2024-05-27T07:04:39.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:38 smithi016 ceph-mon[31415]: pgmap v115: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 73 MiB/s wr, 45 op/s 2024-05-27T07:04:39.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:38 smithi069 ceph-mon[35276]: pgmap v115: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 73 MiB/s wr, 45 op/s 2024-05-27T07:04:41.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:40 smithi164 ceph-mon[35327]: pgmap v116: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 114 MiB/s wr, 70 op/s 2024-05-27T07:04:41.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:40 smithi016 ceph-mon[31415]: pgmap v116: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 114 MiB/s wr, 70 op/s 2024-05-27T07:04:41.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:40 smithi069 ceph-mon[35276]: pgmap v116: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 114 MiB/s wr, 70 op/s 2024-05-27T07:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 -- 172.21.15.16:0/3737108599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 msgr2=0x7f5ff81038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 --2- 172.21.15.16:0/3737108599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff81038c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009990 tx=0x7f5fec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 -- 172.21.15.16:0/3737108599 shutdown_connections 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 --2- 172.21.15.16:0/3737108599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8108730 0x7f5ff806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 --2- 172.21.15.16:0/3737108599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff81038c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 --2- 172.21.15.16:0/3737108599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8107de0 0x7f5ff81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 -- 172.21.15.16:0/3737108599 >> 172.21.15.16:0/3737108599 conn(0x7f5ff80fedb0 msgr2=0x7f5ff81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 -- 172.21.15.16:0/3737108599 shutdown_connections 2024-05-27T07:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.851+0000 7f6000800700 1 -- 172.21.15.16:0/3737108599 wait complete. 2024-05-27T07:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.853+0000 7f6000800700 1 Processor -- start 2024-05-27T07:04:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.853+0000 7f6000800700 1 -- start start 2024-05-27T07:04:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff810fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff81101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81166b0 con 0x7f5ff81034b0 2024-05-27T07:04:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff810bfb0 con 0x7f5ff8107de0 2024-05-27T07:04:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f6000800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8116830 con 0x7f5ff8108730 2024-05-27T07:04:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.854+0000 7f5ffe800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffe800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36626/0 (socket says 172.21.15.16:36626) 2024-05-27T07:04:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffe800700 1 -- 172.21.15.16:0/1484221297 learned_addr learned my addr 172.21.15.16:0/1484221297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffde00700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff810fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffd400700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff81101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffe800700 1 -- 172.21.15.16:0/1484221297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 msgr2=0x7f5ff81101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffe800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff81101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.855+0000 7f5ffe800700 1 -- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 msgr2=0x7f5ff810fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.856+0000 7f5ffe800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff810fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.856+0000 7f5ffe800700 1 -- 172.21.15.16:0/1484221297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff8116110 con 0x7f5ff8108730 2024-05-27T07:04:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.856+0000 7f5ffde00700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff810fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.856+0000 7f5ffe800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b970 tx=0x7f5ff400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.857+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4019070 con 0x7f5ff8108730 2024-05-27T07:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.857+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ff400cdc0 con 0x7f5ff8108730 2024-05-27T07:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.857+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff8116400 con 0x7f5ff8108730 2024-05-27T07:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.857+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a3170 con 0x7f5ff8108730 2024-05-27T07:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.858+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4014e30 con 0x7f5ff8108730 2024-05-27T07:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.861+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc005330 con 0x7f5ff8108730 2024-05-27T07:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.864+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5ff401c070 con 0x7f5ff8108730 2024-05-27T07:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.867+0000 7f5ff2a00700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 0x7f5fe00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.867+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5ff40e1a90 con 0x7f5ff8108730 2024-05-27T07:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.867+0000 7f5ffde00700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 0x7f5fe00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.868+0000 7f5ffde00700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 0x7f5fe00b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b5c0 tx=0x7f5fec005960 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:41.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:41.885+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ff40a91f0 con 0x7f5ff8108730 2024-05-27T07:04:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.380+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fdc005620 con 0x7f5ff8108730 2024-05-27T07:04:42.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.382+0000 7f5ff2a00700 1 -- 172.21.15.16:0/1484221297 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f5ff40b1270 con 0x7f5ff8108730 2024-05-27T07:04:42.374 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:04:42.374 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:04:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 msgr2=0x7f5fe00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 0x7f5fe00b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b5c0 tx=0x7f5fec005960 comp rx=0 tx=0).stop 2024-05-27T07:04:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 msgr2=0x7f5ff8115930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b970 tx=0x7f5ff400bc80 comp rx=0 tx=0).stop 2024-05-27T07:04:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 shutdown_connections 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5fe00b2a70 0x7f5fe00b4f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff8108730 0x7f5ff8115930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff81034b0 0x7f5ff810fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 --2- 172.21.15.16:0/1484221297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff81101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 >> 172.21.15.16:0/1484221297 conn(0x7f5ff80fedb0 msgr2=0x7f5ff8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.386+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 shutdown_connections 2024-05-27T07:04:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:42.387+0000 7f6000800700 1 -- 172.21.15.16:0/1484221297 wait complete. 2024-05-27T07:04:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:04:43.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:42 smithi164 ceph-mon[35327]: pgmap v117: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:43.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:42 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1484221297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:04:43.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:42 smithi016 ceph-mon[31415]: pgmap v117: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:43.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:42 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1484221297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:04:43.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:42 smithi069 ceph-mon[35276]: pgmap v117: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:43.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:42 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1484221297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:04:43.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:04:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:45.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:44 smithi016 ceph-mon[31415]: pgmap v118: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:44 smithi164 ceph-mon[35327]: pgmap v118: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:44 smithi069 ceph-mon[35276]: pgmap v118: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 79 MiB/s wr, 47 op/s 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.553+0000 7f442ae00700 1 -- 172.21.15.16:0/542327897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424110ca0 msgr2=0x7f44241130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.553+0000 7f442ae00700 1 --2- 172.21.15.16:0/542327897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424110ca0 0x7f44241130c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f90 tx=0x7f441c00b330 comp rx=0 tx=0).stop 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.554+0000 7f442ae00700 1 -- 172.21.15.16:0/542327897 shutdown_connections 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.554+0000 7f442ae00700 1 --2- 172.21.15.16:0/542327897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424110ca0 0x7f44241130c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.554+0000 7f442ae00700 1 --2- 172.21.15.16:0/542327897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44241030f0 0x7f4424103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.554+0000 7f442ae00700 1 --2- 172.21.15.16:0/542327897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424103a40 0x7f4424110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.554+0000 7f442ae00700 1 -- 172.21.15.16:0/542327897 >> 172.21.15.16:0/542327897 conn(0x7f44240fedb0 msgr2=0x7f44241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.555+0000 7f442ae00700 1 -- 172.21.15.16:0/542327897 shutdown_connections 2024-05-27T07:04:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.555+0000 7f442ae00700 1 -- 172.21.15.16:0/542327897 wait complete. 2024-05-27T07:04:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.557+0000 7f442ae00700 1 Processor -- start 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.557+0000 7f442ae00700 1 -- start start 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44241030f0 0x7f44241053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 0x7f44241058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 0x7f4424105e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4424116560 con 0x7f44241030f0 2024-05-27T07:04:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44241163e0 con 0x7f4424110ca0 2024-05-27T07:04:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f442ae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44241166e0 con 0x7f4424103a40 2024-05-27T07:04:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.558+0000 7f4423400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 0x7f44241058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4423400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 0x7f44241058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60934/0 (socket says 172.21.15.16:60934) 2024-05-27T07:04:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 0x7f4424105e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4423400700 1 -- 172.21.15.16:0/1797076925 learned_addr learned my addr 172.21.15.16:0/1797076925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 -- 172.21.15.16:0/1797076925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 msgr2=0x7f44241058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 0x7f44241058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 -- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44241030f0 msgr2=0x7f44241053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44241030f0 0x7f44241053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.559+0000 7f4428e00700 1 -- 172.21.15.16:0/1797076925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44240744a0 con 0x7f4424110ca0 2024-05-27T07:04:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.561+0000 7f4428e00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 0x7f4424105e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f90 tx=0x7f441c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.563+0000 7f4420c00700 1 -- 172.21.15.16:0/1797076925 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c021070 con 0x7f4424110ca0 2024-05-27T07:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.563+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4424074730 con 0x7f4424110ca0 2024-05-27T07:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.563+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4424074cc0 con 0x7f4424110ca0 2024-05-27T07:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.566+0000 7f4420c00700 1 -- 172.21.15.16:0/1797076925 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f441c007500 con 0x7f4424110ca0 2024-05-27T07:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.566+0000 7f4420c00700 1 -- 172.21.15.16:0/1797076925 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c021070 con 0x7f4424110ca0 2024-05-27T07:04:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.570+0000 7f44399dab80 1 Processor -- start 2024-05-27T07:04:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.574+0000 7f4420c00700 1 -- 172.21.15.16:0/1797076925 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f441c024070 con 0x7f4424110ca0 2024-05-27T07:04:46.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.575+0000 7f44399dab80 1 -- start start 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff44cfe70 0x558ff44dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff43fde40 con 0x558ff4350e40 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558ff43ed9e0 con 0x558ff44cfe70 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.577+0000 7f44399dab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ff43f1a20 con 0x558ff443c9f0 2024-05-27T07:04:46.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.578+0000 7f4423e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.578+0000 7f4423e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45670/0 (socket says 172.21.15.16:45670) 2024-05-27T07:04:46.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.578+0000 7f4423e00700 1 -- 172.21.15.16:0/2498920552 learned_addr learned my addr 172.21.15.16:0/2498920552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:46.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.579+0000 7f4423400700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff44cfe70 0x558ff44dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.579+0000 7f4423400700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.580+0000 7f4420c00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 0x7f44080b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.582+0000 7f4423400700 1 -- 172.21.15.16:0/2498920552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff44cfe70 msgr2=0x558ff44dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4423400700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff44cfe70 0x558ff44dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4423400700 1 -- 172.21.15.16:0/2498920552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 msgr2=0x558ff44cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4423400700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4423400700 1 -- 172.21.15.16:0/2498920552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff43b6420 con 0x558ff443c9f0 2024-05-27T07:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4423e00700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.584+0000 7f4428e00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 0x7f44080b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.583+0000 7f4420c00700 1 -- 172.21.15.16:0/1797076925 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f441c00f020 con 0x7f4424110ca0 2024-05-27T07:04:46.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.584+0000 7f4423400700 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44cf330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4414010940 tx=0x7f4414009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb95b1b7d351f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.586+0000 7f4428e00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 0x7f44080b51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f441c0a68f0 tx=0x7f441c0a6040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.588+0000 7f4419600700 1 -- 172.21.15.16:0/2498920552 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414016030 con 0x558ff443c9f0 2024-05-27T07:04:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.588+0000 7f4419600700 1 -- 172.21.15.16:0/2498920552 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414005020 con 0x558ff443c9f0 2024-05-27T07:04:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.588+0000 7f4419600700 1 -- 172.21.15.16:0/2498920552 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019040 con 0x558ff443c9f0 2024-05-27T07:04:46.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.589+0000 7f44399dab80 1 -- 172.21.15.16:0/2498920552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 msgr2=0x558ff44cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.589+0000 7f44399dab80 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44cf330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4414010940 tx=0x7f4414009310 comp rx=0 tx=0).stop 2024-05-27T07:04:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.590+0000 7f44399dab80 1 -- 172.21.15.16:0/2498920552 shutdown_connections 2024-05-27T07:04:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.590+0000 7f44399dab80 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff44cfe70 0x558ff44dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.590+0000 7f44399dab80 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44cf330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.590+0000 7f44399dab80 1 --2- 172.21.15.16:0/2498920552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.590+0000 7f44399dab80 1 -- 172.21.15.16:0/2498920552 >> 172.21.15.16:0/2498920552 conn(0x558ff43ea390 msgr2=0x558ff44cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.591+0000 7f44399dab80 1 -- 172.21.15.16:0/2498920552 shutdown_connections 2024-05-27T07:04:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.591+0000 7f44399dab80 1 -- 172.21.15.16:0/2498920552 wait complete. 2024-05-27T07:04:46.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.592+0000 7f44399dab80 1 -- start start 2024-05-27T07:04:46.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.592+0000 7f44399dab80 1 Processor -- start 2024-05-27T07:04:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff42f3fa0 con 0x558ff4350e40 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558ff438e760 con 0x558ff443c9f0 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ff44dcb80 con 0x558ff44cfe70 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.595+0000 7f44399dab80 10 client.0 _ll_register_callbacks cb 0x558ff4397c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.596+0000 7f4423e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.596+0000 7f4423e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42432/0 (socket says 172.21.15.16:42432) 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.596+0000 7f4423e00700 1 -- 172.21.15.16:0/3316816479 learned_addr learned my addr 172.21.15.16:0/3316816479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.597+0000 7f4423e00700 1 -- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 msgr2=0x558ff44da1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.597+0000 7f4423e00700 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44da1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.597+0000 7f4423e00700 1 -- 172.21.15.16:0/3316816479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 msgr2=0x558ff44d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.597+0000 7f4423e00700 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.597+0000 7f4423e00700 1 -- 172.21.15.16:0/3316816479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff433d8d0 con 0x558ff443c9f0 2024-05-27T07:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.598+0000 7f4423e00700 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f441000bef0 tx=0x7f44100106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7d8141d36c8508 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.598+0000 7f440e000700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.600+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410004250 con 0x558ff443c9f0 2024-05-27T07:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.600+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f44100043f0 con 0x558ff443c9f0 2024-05-27T07:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.600+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4414005020 con 0x558ff443c9f0 2024-05-27T07:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.601+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4410003680 con 0x558ff443c9f0 2024-05-27T07:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.601+0000 7f44399dab80 10 client.35765 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:46.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.602+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558ff43be060 con 0x558ff443c9f0 2024-05-27T07:04:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.603+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414005020 con 0x558ff443c9f0 2024-05-27T07:04:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.606+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f441001a8e0 con 0x558ff443c9f0 2024-05-27T07:04:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.607+0000 7f44399dab80 10 client.35765 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.607+0000 7f44399dab80 10 client.35765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.608+0000 7f44399dab80 20 client.35765 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.608+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff44e9ae0 0x558ff44ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.608+0000 7f44399dab80 4 client.35765 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.608+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff43ffd90 con 0x558ff44e9ae0 2024-05-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.609+0000 7f4423400700 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff44e9ae0 0x558ff44ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.612+0000 7f4423400700 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff44e9ae0 0x558ff44ebf90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2dfd3c57e17829 server_cookie=9a21584c9dd556e7 in_seq=0 out_seq=0 2024-05-27T07:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.613+0000 7f440e000700 10 client.35765 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f440e000700 1 -- 172.21.15.16:0/3316816479 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558ff43ffd90 con 0x558ff44e9ae0 2024-05-27T07:04:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f440e000700 10 client.35765 handle_command_reply: tid=0 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f44399dab80 1 client.35765 shutdown 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 msgr2=0x558ff44d9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f441000bef0 tx=0x7f44100106b0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 shutdown_connections 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.615+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff44e9ae0 0x558ff44ebf90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44da1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f44399dab80 1 --2- 172.21.15.16:0/3316816479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d9c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f440e000700 0 client.35765 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 >> 172.21.15.16:0/3316816479 conn(0x558ff43ea390 msgr2=0x558ff44cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.616+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 shutdown_connections 2024-05-27T07:04:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.617+0000 7f44399dab80 1 -- 172.21.15.16:0/3316816479 wait complete. 2024-05-27T07:04:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.617+0000 7f44399dab80 20 client.35765 trim_cache size 0 max 16384 2024-05-27T07:04:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.650+0000 7f44399dab80 1 Processor -- start 2024-05-27T07:04:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 -- start start 2024-05-27T07:04:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 0x558ff44d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff44cfe70 0x558ff44cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f440400ee00 con 0x558ff44cfe70 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4404010940 con 0x558ff4350e40 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.651+0000 7f44399dab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f440400f4d0 con 0x558ff443c9f0 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.652+0000 7f4423e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.652+0000 7f4423e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60970/0 (socket says 172.21.15.16:60970) 2024-05-27T07:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.652+0000 7f4423e00700 1 -- 172.21.15.16:0/2645158871 learned_addr learned my addr 172.21.15.16:0/2645158871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.653+0000 7f4423400700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff44cfe70 0x558ff44cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423400700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 0x558ff44d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423e00700 1 -- 172.21.15.16:0/2645158871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 msgr2=0x558ff44d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423e00700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 0x558ff44d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423e00700 1 -- 172.21.15.16:0/2645158871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff44cfe70 msgr2=0x558ff44cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423e00700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff44cfe70 0x558ff44cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423e00700 1 -- 172.21.15.16:0/2645158871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff44d7c90 con 0x558ff443c9f0 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.654+0000 7f4423400700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 0x558ff44d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.655+0000 7f4423e00700 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f441000bec0 tx=0x7f4410005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce93c0e8d27062bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.656+0000 7f4418c00700 1 -- 172.21.15.16:0/2645158871 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410004a30 con 0x558ff443c9f0 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.656+0000 7f4418c00700 1 -- 172.21.15.16:0/2645158871 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4410015070 con 0x558ff443c9f0 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.656+0000 7f4418c00700 1 -- 172.21.15.16:0/2645158871 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100117b0 con 0x558ff443c9f0 2024-05-27T07:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.657+0000 7f44399dab80 1 -- 172.21.15.16:0/2645158871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 msgr2=0x558ff44d4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.657+0000 7f44399dab80 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f441000bec0 tx=0x7f4410005a00 comp rx=0 tx=0).stop 2024-05-27T07:04:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 -- 172.21.15.16:0/2645158871 shutdown_connections 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff44cfe70 0x558ff44cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff4350e40 0x558ff44d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 --2- 172.21.15.16:0/2645158871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff443c9f0 0x558ff44d4ef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 -- 172.21.15.16:0/2645158871 >> 172.21.15.16:0/2645158871 conn(0x558ff43ea390 msgr2=0x558ff44cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.658+0000 7f44399dab80 1 -- 172.21.15.16:0/2645158871 shutdown_connections 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.659+0000 7f44399dab80 1 -- 172.21.15.16:0/2645158871 wait complete. 2024-05-27T07:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.659+0000 7f44399dab80 1 -- start start 2024-05-27T07:04:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.660+0000 7f44399dab80 1 Processor -- start 2024-05-27T07:04:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4404010270 con 0x558ff4350e40 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f440400fba0 con 0x558ff443c9f0 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f440400cda0 con 0x558ff44cfe70 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.662+0000 7f44399dab80 10 client.0 _ll_register_callbacks cb 0x558ff4397c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.663+0000 7f4423400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.663+0000 7f4423e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.664+0000 7f4423400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45712/0 (socket says 172.21.15.16:45712) 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.664+0000 7f4423400700 1 -- 172.21.15.16:0/1051390470 learned_addr learned my addr 172.21.15.16:0/1051390470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.664+0000 7f4423400700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423e00700 1 -- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 msgr2=0x558ff44e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423e00700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423e00700 1 -- 172.21.15.16:0/1051390470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 msgr2=0x558ff44d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423e00700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423e00700 1 -- 172.21.15.16:0/1051390470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff44d4510 con 0x558ff443c9f0 2024-05-27T07:04:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.665+0000 7f4423400700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.666+0000 7f440e000700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.667+0000 7f4423e00700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d3c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f441001ace0 tx=0x7f4410004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd84b436b0970c7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.667+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441001a680 con 0x558ff443c9f0 2024-05-27T07:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.667+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4410032070 con 0x558ff443c9f0 2024-05-27T07:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.668+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4410015070 con 0x558ff443c9f0 2024-05-27T07:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.668+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f441001a680 con 0x558ff443c9f0 2024-05-27T07:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.669+0000 7f44399dab80 10 client.35768 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.669+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558ff44e2370 con 0x558ff443c9f0 2024-05-27T07:04:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4410015070 con 0x558ff443c9f0 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4410011760 con 0x558ff443c9f0 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 10 client.35768 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 10 client.35768 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 20 client.35768 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff45546b0 0x558ff4556ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 4 client.35768 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.670+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff43f9570 con 0x558ff45546b0 2024-05-27T07:04:46.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.673+0000 7f4423400700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff45546b0 0x558ff4556ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.675+0000 7f440e000700 10 client.35768 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.675+0000 7f4423400700 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff45546b0 0x558ff4556ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c626a47f0d5df2b server_cookie=539ba0454e0140f3 in_seq=0 out_seq=0 2024-05-27T07:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.675+0000 7f440e000700 1 -- 172.21.15.16:0/1051390470 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff43f9570 con 0x558ff45546b0 2024-05-27T07:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.675+0000 7f440e000700 10 client.35768 handle_command_reply: tid=0 2024-05-27T07:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.675+0000 7f44399dab80 1 client.35768 shutdown 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 msgr2=0x558ff44d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d3c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f441001ace0 tx=0x7f4410004a60 comp rx=0 tx=0).stop 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 shutdown_connections 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x558ff45546b0 0x558ff4556ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f440e000700 0 client.35768 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff44cfe70 0x558ff44e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff4350e40 0x558ff44d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 --2- 172.21.15.16:0/1051390470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ff443c9f0 0x558ff44d3c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.676+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 >> 172.21.15.16:0/1051390470 conn(0x558ff43ea390 msgr2=0x558ff44d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.677+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 shutdown_connections 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.677+0000 7f44399dab80 1 -- 172.21.15.16:0/1051390470 wait complete. 2024-05-27T07:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.677+0000 7f44399dab80 20 client.35768 trim_cache size 0 max 16384 2024-05-27T07:04:46.669 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.679+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 msgr2=0x7f44080b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.679+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 0x7f44080b51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f441c0a68f0 tx=0x7f441c0a6040 comp rx=0 tx=0).stop 2024-05-27T07:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.679+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 msgr2=0x7f4424105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.679+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 0x7f4424105e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f90 tx=0x7f441c00cd70 comp rx=0 tx=0).stop 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 shutdown_connections 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f44080b2ca0 0x7f44080b51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4424110ca0 0x7f4424105e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44241030f0 0x7f44241053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 --2- 172.21.15.16:0/1797076925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4424103a40 0x7f44241058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 >> 172.21.15.16:0/1797076925 conn(0x7f44240fedb0 msgr2=0x7f44241127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 shutdown_connections 2024-05-27T07:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:46.680+0000 7f442ae00700 1 -- 172.21.15.16:0/1797076925 wait complete. 2024-05-27T07:04:47.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:46 smithi164 ceph-mon[35327]: pgmap v119: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:46 smithi016 ceph-mon[31415]: pgmap v119: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:47.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:46 smithi069 ceph-mon[35276]: pgmap v119: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 120 MiB/s wr, 73 op/s 2024-05-27T07:04:47.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:04:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:49.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:48 smithi164 ceph-mon[35327]: pgmap v120: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 83 MiB/s wr, 50 op/s 2024-05-27T07:04:49.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:48 smithi016 ceph-mon[31415]: pgmap v120: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 83 MiB/s wr, 50 op/s 2024-05-27T07:04:49.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:48 smithi069 ceph-mon[35276]: pgmap v120: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 83 MiB/s wr, 50 op/s 2024-05-27T07:04:50.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:50 smithi016 ceph-mon[31415]: pgmap v121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 108 MiB/s wr, 230 op/s 2024-05-27T07:04:51.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:50 smithi069 ceph-mon[35276]: pgmap v121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 108 MiB/s wr, 230 op/s 2024-05-27T07:04:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:50 smithi164 ceph-mon[35327]: pgmap v121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 108 MiB/s wr, 230 op/s 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.781+0000 7fa23c400700 1 -- 172.21.15.16:0/4109724732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa234107de0 msgr2=0x7fa2341081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.781+0000 7fa23c400700 1 --2- 172.21.15.16:0/4109724732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa234107de0 0x7fa2341081f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2200098f0 tx=0x7fa220009c00 comp rx=0 tx=0).stop 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.782+0000 7fa23c400700 1 -- 172.21.15.16:0/4109724732 shutdown_connections 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.782+0000 7fa23c400700 1 --2- 172.21.15.16:0/4109724732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 0x7fa234110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.782+0000 7fa23c400700 1 --2- 172.21.15.16:0/4109724732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2341034b0 0x7fa2341038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.782+0000 7fa23c400700 1 --2- 172.21.15.16:0/4109724732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa234107de0 0x7fa2341081f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.782+0000 7fa23c400700 1 -- 172.21.15.16:0/4109724732 >> 172.21.15.16:0/4109724732 conn(0x7fa2340fedb0 msgr2=0x7fa2341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.784+0000 7fa23c400700 1 -- 172.21.15.16:0/4109724732 shutdown_connections 2024-05-27T07:04:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.784+0000 7fa23c400700 1 -- 172.21.15.16:0/4109724732 wait complete. 2024-05-27T07:04:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.787+0000 7fa23c400700 1 Processor -- start 2024-05-27T07:04:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.787+0000 7fa23c400700 1 -- start start 2024-05-27T07:04:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 0x7fa2340753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 0x7fa23406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa234112160 con 0x7fa234108730 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa234111fe0 con 0x7fa234107de0 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.788+0000 7fa23c400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341122e0 con 0x7fa2341034b0 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.789+0000 7fa239000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.789+0000 7fa239000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42466/0 (socket says 172.21.15.16:42466) 2024-05-27T07:04:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.789+0000 7fa239000700 1 -- 172.21.15.16:0/1979285747 learned_addr learned my addr 172.21.15.16:0/1979285747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.789+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 0x7fa2340753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.789+0000 7fa23a400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 0x7fa23406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.790+0000 7fa239a00700 1 -- 172.21.15.16:0/1979285747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 msgr2=0x7fa23406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.790+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.790+0000 7fa239a00700 1 -- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 msgr2=0x7fa23406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.790+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 0x7fa23406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.790+0000 7fa239a00700 1 -- 172.21.15.16:0/1979285747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa234072f20 con 0x7fa2341034b0 2024-05-27T07:04:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.791+0000 7fa239000700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.791+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 0x7fa2340753f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22800e9f0 tx=0x7fa22800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.792+0000 7fa232000700 1 -- 172.21.15.16:0/1979285747 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800c9c0 con 0x7fa2341034b0 2024-05-27T07:04:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.792+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2340731b0 con 0x7fa2341034b0 2024-05-27T07:04:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.792+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa234073740 con 0x7fa2341034b0 2024-05-27T07:04:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.793+0000 7fa232000700 1 -- 172.21.15.16:0/1979285747 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa228010430 con 0x7fa2341034b0 2024-05-27T07:04:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.793+0000 7fa232000700 1 -- 172.21.15.16:0/1979285747 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228018680 con 0x7fa2341034b0 2024-05-27T07:04:51.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.802+0000 7fa24af01b80 1 Processor -- start 2024-05-27T07:04:51.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 -- start start 2024-05-27T07:04:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea3c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f11e96ae40 con 0x55f11e9a99f0 2024-05-27T07:04:51.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f11e95a9e0 con 0x55f11e8bde40 2024-05-27T07:04:51.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.803+0000 7fa24af01b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f11e95ea20 con 0x55f11ea3ce70 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.804+0000 7fa23a400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.804+0000 7fa232000700 1 -- 172.21.15.16:0/1979285747 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa22800cb60 con 0x7fa2341034b0 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.805+0000 7fa23a400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45746/0 (socket says 172.21.15.16:45746) 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.805+0000 7fa23a400700 1 -- 172.21.15.16:0/2065764050 learned_addr learned my addr 172.21.15.16:0/2065764050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.805+0000 7fa239a00700 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea3c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa23a400700 1 -- 172.21.15.16:0/2065764050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 msgr2=0x55f11ea49110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa23a400700 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa23a400700 1 -- 172.21.15.16:0/2065764050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 msgr2=0x55f11ea3c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa23a400700 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea3c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa23a400700 1 -- 172.21.15.16:0/2065764050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11e923420 con 0x55f11e9a99f0 2024-05-27T07:04:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.806+0000 7fa239a00700 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea3c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:04:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.807+0000 7fa23a400700 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011600 tx=0x7fa22c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fc269a2ca597fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.808+0000 7fa232000700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 0x7fa21c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.808+0000 7fa226a00700 1 -- 172.21.15.16:0/2065764050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c019030 con 0x55f11e9a99f0 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.808+0000 7fa226a00700 1 -- 172.21.15.16:0/2065764050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c0092e0 con 0x55f11e9a99f0 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.808+0000 7fa226a00700 1 -- 172.21.15.16:0/2065764050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0047a0 con 0x55f11e9a99f0 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.809+0000 7fa232000700 1 -- 172.21.15.16:0/1979285747 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa2280e2a30 con 0x7fa2341034b0 2024-05-27T07:04:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.809+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 0x7fa21c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 -- 172.21.15.16:0/2065764050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 msgr2=0x55f11ea3c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011600 tx=0x7fa22c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 -- 172.21.15.16:0/2065764050 shutdown_connections 2024-05-27T07:04:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea3c330 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2065764050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea3c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.810+0000 7fa24af01b80 1 -- 172.21.15.16:0/2065764050 >> 172.21.15.16:0/2065764050 conn(0x55f11e957390 msgr2=0x55f11ea384c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.812+0000 7fa24af01b80 1 -- 172.21.15.16:0/2065764050 shutdown_connections 2024-05-27T07:04:51.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.812+0000 7fa239a00700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 0x7fa21c0b5000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f11ea39a20 tx=0x7fa2280a96f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.812+0000 7fa24af01b80 1 -- 172.21.15.16:0/2065764050 wait complete. 2024-05-27T07:04:51.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.813+0000 7fa24af01b80 1 -- start start 2024-05-27T07:04:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.814+0000 7fa24af01b80 1 Processor -- start 2024-05-27T07:04:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea46640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f11e860fa0 con 0x55f11e8bde40 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f11e8fb760 con 0x55f11e9a99f0 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f11ea49b80 con 0x55f11ea3ce70 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.815+0000 7fa24af01b80 10 client.0 _ll_register_callbacks cb 0x55f11e904c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.816+0000 7fa23a400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea46640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.816+0000 7fa23a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.816+0000 7fa239000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.816+0000 7fa239000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42482/0 (socket says 172.21.15.16:42482) 2024-05-27T07:04:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.816+0000 7fa239000700 1 -- 172.21.15.16:0/526320103 learned_addr learned my addr 172.21.15.16:0/526320103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.817+0000 7fa23a400700 1 -- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 msgr2=0x55f11ea46b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.817+0000 7fa23a400700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.817+0000 7fa23a400700 1 -- 172.21.15.16:0/526320103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 msgr2=0x55f11ea46640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.817+0000 7fa23a400700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea46640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.818+0000 7fa23a400700 1 -- 172.21.15.16:0/526320103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11e8aa8d0 con 0x55f11ea3ce70 2024-05-27T07:04:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.818+0000 7fa239000700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:04:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.819+0000 7fa23a400700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea470c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0115d0 tx=0x7fa22c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b90336401fd514 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.819+0000 7fa21b400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:04:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.820+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0048f0 con 0x55f11ea3ce70 2024-05-27T07:04:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.820+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22c0092e0 con 0x55f11ea3ce70 2024-05-27T07:04:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.820+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa22c004ad0 con 0x55f11ea3ce70 2024-05-27T07:04:51.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.822+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa22c004cd0 con 0x55f11ea3ce70 2024-05-27T07:04:51.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.822+0000 7fa24af01b80 10 client.25726 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:51.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.822+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f11e92b060 con 0x55f11ea3ce70 2024-05-27T07:04:51.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c0092e0 con 0x55f11ea3ce70 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa22c015450 con 0x55f11ea3ce70 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 10 client.25726 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 10 client.25726 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 20 client.25726 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11ea56a10 0x55f11ea58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 4 client.25726 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:04:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.825+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11e96cd90 con 0x55f11ea56a10 2024-05-27T07:04:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.826+0000 7fa239000700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11ea56a10 0x55f11ea58e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.828+0000 7fa239000700 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11ea56a10 0x55f11ea58e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a45a142b9a5eccd9 server_cookie=32f77e77e142ae44 in_seq=0 out_seq=0 2024-05-27T07:04:51.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.828+0000 7fa21b400700 10 client.25726 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:51.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.830+0000 7fa21b400700 1 -- 172.21.15.16:0/526320103 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f11e96cd90 con 0x55f11ea56a10 2024-05-27T07:04:51.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.830+0000 7fa21b400700 10 client.25726 handle_command_reply: tid=0 2024-05-27T07:04:51.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.830+0000 7fa24af01b80 1 client.25726 shutdown 2024-05-27T07:04:51.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.830+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 msgr2=0x55f11ea470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.830+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea470c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0115d0 tx=0x7fa22c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 shutdown_connections 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea470c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea46640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11ea56a10 0x55f11ea58e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 --2- 172.21.15.16:0/526320103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa21b400700 0 client.25726 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.832+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 >> 172.21.15.16:0/526320103 conn(0x55f11e957390 msgr2=0x55f11ea3c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.833+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 shutdown_connections 2024-05-27T07:04:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.833+0000 7fa24af01b80 1 -- 172.21.15.16:0/526320103 wait complete. 2024-05-27T07:04:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.833+0000 7fa24af01b80 20 client.25726 trim_cache size 0 max 16384 2024-05-27T07:04:51.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.874+0000 7fa24af01b80 1 Processor -- start 2024-05-27T07:04:51.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 -- start start 2024-05-27T07:04:51.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa21400ed20 con 0x55f11e9a99f0 2024-05-27T07:04:51.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa214010860 con 0x55f11e8bde40 2024-05-27T07:04:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.875+0000 7fa24af01b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa21400f3f0 con 0x55f11ea3ce70 2024-05-27T07:04:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.876+0000 7fa23a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.877+0000 7fa23a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-05-27T07:04:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.877+0000 7fa23a400700 1 -- 172.21.15.16:0/3730245933 learned_addr learned my addr 172.21.15.16:0/3730245933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.877+0000 7fa239000700 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.877+0000 7fa23a400700 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.879+0000 7fa23a400700 1 -- 172.21.15.16:0/3730245933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 msgr2=0x55f11ea455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.879+0000 7fa23a400700 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.879+0000 7fa23a400700 1 -- 172.21.15.16:0/3730245933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 msgr2=0x55f11ea405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.879+0000 7fa23a400700 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.879+0000 7fa23a400700 1 -- 172.21.15.16:0/3730245933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11ea39850 con 0x55f11ea3ce70 2024-05-27T07:04:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.881+0000 7fa23a400700 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa22c004f90 tx=0x7fa22c006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65ebfcd538cb439 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.882+0000 7fa226000700 1 -- 172.21.15.16:0/3730245933 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c019020 con 0x55f11ea3ce70 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.882+0000 7fa226000700 1 -- 172.21.15.16:0/3730245933 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c009a50 con 0x55f11ea3ce70 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.882+0000 7fa226000700 1 -- 172.21.15.16:0/3730245933 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c004a30 con 0x55f11ea3ce70 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 -- 172.21.15.16:0/3730245933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 msgr2=0x55f11ea52c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa22c004f90 tx=0x7fa22c006000 comp rx=0 tx=0).stop 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 -- 172.21.15.16:0/3730245933 shutdown_connections 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea52c80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e8bde40 0x55f11ea455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 --2- 172.21.15.16:0/3730245933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e9a99f0 0x55f11ea405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.884+0000 7fa24af01b80 1 -- 172.21.15.16:0/3730245933 >> 172.21.15.16:0/3730245933 conn(0x55f11e957390 msgr2=0x55f11ea3fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.885+0000 7fa24af01b80 1 -- 172.21.15.16:0/3730245933 shutdown_connections 2024-05-27T07:04:51.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.885+0000 7fa24af01b80 1 -- 172.21.15.16:0/3730245933 wait complete. 2024-05-27T07:04:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.886+0000 7fa24af01b80 1 -- start start 2024-05-27T07:04:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.886+0000 7fa24af01b80 1 Processor -- start 2024-05-27T07:04:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.888+0000 7fa24af01b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.889+0000 7fa23a400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.889+0000 7fa23a400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea49e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50830/0 (socket says 172.21.15.16:50830) 2024-05-27T07:04:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.889+0000 7fa23a400700 1 -- 172.21.15.16:0/2204037101 learned_addr learned my addr 172.21.15.16:0/2204037101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:04:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.889+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea4a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.890+0000 7fa239000700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea4a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.890+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea4a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.890+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa214010190 con 0x55f11e8bde40 2024-05-27T07:04:51.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.891+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa21400fac0 con 0x55f11e9a99f0 2024-05-27T07:04:51.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.891+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa21400ccc0 con 0x55f11ea3ce70 2024-05-27T07:04:51.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.891+0000 7fa24af01b80 10 client.0 _ll_register_callbacks cb 0x55f11e904c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:04:51.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.893+0000 7fa239000700 1 -- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 msgr2=0x55f11ea4a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.893+0000 7fa23a400700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea4a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.894+0000 7fa239000700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea4a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.894+0000 7fa239000700 1 -- 172.21.15.16:0/2204037101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 msgr2=0x55f11ea49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:04:51.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.894+0000 7fa239000700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.894+0000 7fa239000700 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11eabf350 con 0x55f11e9a99f0 2024-05-27T07:04:51.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.894+0000 7fa23a400700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:04:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.898+0000 7fa239000700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea4a370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f11ea32420 tx=0x7fa220005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db034629fbdd3748 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:04:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.898+0000 7fa21b400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:04:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.899+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220005520 con 0x55f11e9a99f0 2024-05-27T07:04:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.899+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22c009a50 con 0x55f11e9a99f0 2024-05-27T07:04:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.899+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa220003680 con 0x55f11e9a99f0 2024-05-27T07:04:51.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.900+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2200056c0 con 0x55f11e9a99f0 2024-05-27T07:04:51.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.901+0000 7fa24af01b80 10 client.35786 fetch_fsmap learned FSMap version 31 2024-05-27T07:04:51.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.901+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f11eac0590 con 0x55f11e9a99f0 2024-05-27T07:04:51.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.901+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c009a50 con 0x55f11e9a99f0 2024-05-27T07:04:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa22000ba90 con 0x55f11e9a99f0 2024-05-27T07:04:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 10 client.35786 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 10 client.35786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 20 client.35786 populate_metadata read hostname 'smithi016' 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11eac0ca0 0x55f11eac30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 4 client.35786 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.902+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f11e966570 con 0x55f11eac0ca0 2024-05-27T07:04:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.903+0000 7fa23a400700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11eac0ca0 0x55f11eac30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:04:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.905+0000 7fa23a400700 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11eac0ca0 0x55f11eac30c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8bba098216a6a072 server_cookie=abc5421097d647a7 in_seq=0 out_seq=0 2024-05-27T07:04:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.905+0000 7fa21b400700 10 client.35786 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.905+0000 7fa21b400700 1 -- 172.21.15.16:0/2204037101 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f11e966570 con 0x55f11eac0ca0 2024-05-27T07:04:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.905+0000 7fa21b400700 10 client.35786 handle_command_reply: tid=0 2024-05-27T07:04:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.906+0000 7fa24af01b80 1 client.35786 shutdown 2024-05-27T07:04:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.907+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 msgr2=0x55f11ea4a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.907+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea4a370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f11ea32420 tx=0x7fa220005040 comp rx=0 tx=0).stop 2024-05-27T07:04:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 shutdown_connections 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f11ea3ce70 0x55f11ea4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f11e8bde40 0x55f11ea49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55f11eac0ca0 0x55f11eac30c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 --2- 172.21.15.16:0/2204037101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f11e9a99f0 0x55f11ea4a370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 >> 172.21.15.16:0/2204037101 conn(0x55f11e957390 msgr2=0x55f11ea46e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa21b400700 0 client.35786 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:04:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.908+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 shutdown_connections 2024-05-27T07:04:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.909+0000 7fa24af01b80 1 -- 172.21.15.16:0/2204037101 wait complete. 2024-05-27T07:04:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.909+0000 7fa24af01b80 20 client.35786 trim_cache size 0 max 16384 2024-05-27T07:04:51.901 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.911+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 msgr2=0x7fa21c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.911+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 0x7fa21c0b5000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f11ea39a20 tx=0x7fa2280a96f0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.911+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 msgr2=0x7fa2340753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.911+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 0x7fa2340753f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22800e9f0 tx=0x7fa22800ed00 comp rx=0 tx=0).stop 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 shutdown_connections 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa21c0b2af0 0x7fa21c0b5000 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234108730 0x7fa23406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2341034b0 0x7fa2340753f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 --2- 172.21.15.16:0/1979285747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa234107de0 0x7fa23406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:04:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 >> 172.21.15.16:0/1979285747 conn(0x7fa2340fedb0 msgr2=0x7fa2341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:04:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.912+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 shutdown_connections 2024-05-27T07:04:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:04:51.914+0000 7fa23c400700 1 -- 172.21.15.16:0/1979285747 wait complete. 2024-05-27T07:04:53.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:52 smithi016 ceph-mon[31415]: pgmap v122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:53.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:52 smithi069 ceph-mon[35276]: pgmap v122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:52 smithi164 ceph-mon[35327]: pgmap v122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:54.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:04:55.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:54 smithi016 ceph-mon[31415]: pgmap v123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:55.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:54 smithi069 ceph-mon[35276]: pgmap v123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:55.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:54 smithi164 ceph-mon[35327]: pgmap v123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 66 MiB/s wr, 204 op/s 2024-05-27T07:04:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:04:57.133 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:56 smithi164 ceph-mon[35327]: pgmap v124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 81 MiB/s wr, 454 op/s 2024-05-27T07:04:57.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:56 smithi016 ceph-mon[31415]: pgmap v124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 81 MiB/s wr, 454 op/s 2024-05-27T07:04:57.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:56 smithi069 ceph-mon[35276]: pgmap v124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 81 MiB/s wr, 454 op/s 2024-05-27T07:04:59.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:04:58 smithi016 ceph-mon[31415]: pgmap v125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 40 MiB/s wr, 429 op/s 2024-05-27T07:04:59.171 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:04:58 smithi069 ceph-mon[35276]: pgmap v125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 40 MiB/s wr, 429 op/s 2024-05-27T07:04:59.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:04:58 smithi164 ceph-mon[35327]: pgmap v125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 40 MiB/s wr, 429 op/s 2024-05-27T07:05:01.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:00 smithi164 ceph-mon[35327]: pgmap v126: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 685 op/s 2024-05-27T07:05:01.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:00 smithi016 ceph-mon[31415]: pgmap v126: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 685 op/s 2024-05-27T07:05:01.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:00 smithi069 ceph-mon[35276]: pgmap v126: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 685 op/s 2024-05-27T07:05:03.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:02 smithi016 ceph-mon[31415]: pgmap v127: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:03.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:02 smithi164 ceph-mon[35327]: pgmap v127: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:03.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:02 smithi069 ceph-mon[35276]: pgmap v127: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.901+0000 7f98f2800700 1 -- 172.21.15.16:0/748041413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec107be0 msgr2=0x7f98ec107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.901+0000 7f98f2800700 1 --2- 172.21.15.16:0/748041413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec107be0 0x7f98ec107ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98e00099c0 tx=0x7f98e0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.903+0000 7f98f2800700 1 -- 172.21.15.16:0/748041413 shutdown_connections 2024-05-27T07:05:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.903+0000 7f98f2800700 1 --2- 172.21.15.16:0/748041413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec108530 0x7f98ec10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.903+0000 7f98f2800700 1 --2- 172.21.15.16:0/748041413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1032b0 0x7f98ec1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.903+0000 7f98f2800700 1 --2- 172.21.15.16:0/748041413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec107be0 0x7f98ec107ff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.903+0000 7f98f2800700 1 -- 172.21.15.16:0/748041413 >> 172.21.15.16:0/748041413 conn(0x7f98ec078be0 msgr2=0x7f98ec079020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.904+0000 7f98f2800700 1 -- 172.21.15.16:0/748041413 shutdown_connections 2024-05-27T07:05:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.904+0000 7f98f2800700 1 -- 172.21.15.16:0/748041413 wait complete. 2024-05-27T07:05:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.907+0000 7f98f2800700 1 Processor -- start 2024-05-27T07:05:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.907+0000 7f98f2800700 1 -- start start 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.908+0000 7f98f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.908+0000 7f98f2800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.909+0000 7f98f1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.910+0000 7f98f1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:45710/0 (socket says 172.21.15.16:45710) 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f2800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec108530 0x7f98ec10cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f2800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec111f90 con 0x7f98ec108530 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f2800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec111e10 con 0x7f98ec107be0 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f2800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec112110 con 0x7f98ec1032b0 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34454/0 (socket says 172.21.15.16:34454) 2024-05-27T07:05:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.911+0000 7f98f0a00700 1 -- 172.21.15.16:0/1321055072 learned_addr learned my addr 172.21.15.16:0/1321055072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f0a00700 1 -- 172.21.15.16:0/1321055072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 msgr2=0x7f98ec10c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f0a00700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f0a00700 1 -- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec108530 msgr2=0x7f98ec10cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f0a00700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec108530 0x7f98ec10cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f0a00700 1 -- 172.21.15.16:0/1321055072 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec1ad2a0 con 0x7f98ec107be0 2024-05-27T07:05:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.912+0000 7f98f1400700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-27T07:05:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.913+0000 7f98f0a00700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098e0 tx=0x7f98e000f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.915+0000 7f98ea000700 1 -- 172.21.15.16:0/1321055072 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0026070 con 0x7f98ec107be0 2024-05-27T07:05:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.916+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec1ad530 con 0x7f98ec107be0 2024-05-27T07:05:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.916+0000 7f98ea000700 1 -- 172.21.15.16:0/1321055072 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e000fd80 con 0x7f98ec107be0 2024-05-27T07:05:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.916+0000 7f98ea000700 1 -- 172.21.15.16:0/1321055072 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00173f0 con 0x7f98ec107be0 2024-05-27T07:05:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.916+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec1ad9e0 con 0x7f98ec107be0 2024-05-27T07:05:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.928+0000 7f98ea000700 1 -- 172.21.15.16:0/1321055072 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f98e000f8f0 con 0x7f98ec107be0 2024-05-27T07:05:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.931+0000 7f99028e4b80 1 Processor -- start 2024-05-27T07:05:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.931+0000 7f99028e4b80 1 -- start start 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.934+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.934+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 0x560c0abf1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.934+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 0x560c0abf4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.935+0000 7f99028e4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560c0ab15e40 con 0x560c0aa68e40 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.935+0000 7f99028e4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c0ab059e0 con 0x560c0abe7410 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.935+0000 7f99028e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c0ab09a20 con 0x560c0abe7820 2024-05-27T07:05:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.935+0000 7f98f1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 0x560c0abf1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.935+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 0x560c0abf4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.936+0000 7f98f1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 0x560c0abf1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34462/0 (socket says 172.21.15.16:34462) 2024-05-27T07:05:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.936+0000 7f98f1400700 1 -- 172.21.15.16:0/3418913310 learned_addr learned my addr 172.21.15.16:0/3418913310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.936+0000 7f98ea000700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 0x7f98d40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.937+0000 7f98f1e00700 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98ea000700 1 -- 172.21.15.16:0/1321055072 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f98e00e3440 con 0x7f98ec107be0 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98f0a00700 1 -- 172.21.15.16:0/3418913310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 msgr2=0x560c0abf1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98f0a00700 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 0x560c0abf1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98f0a00700 1 -- 172.21.15.16:0/3418913310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 msgr2=0x560c0abe6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98f0a00700 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abe6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.938+0000 7f98f0a00700 1 -- 172.21.15.16:0/3418913310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0aace420 con 0x560c0abe7820 2024-05-27T07:05:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.939+0000 7f98f0a00700 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 0x560c0abf4020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98e00e0800 tx=0x7f98e0021530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae89d708fcfda56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.939+0000 7f98f1e00700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 0x7f98d40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.939+0000 7f98dea00700 1 -- 172.21.15.16:0/3418913310 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00a62e0 con 0x560c0abe7820 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.940+0000 7f98dea00700 1 -- 172.21.15.16:0/3418913310 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e00a9f30 con 0x560c0abe7820 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 -- 172.21.15.16:0/3418913310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 msgr2=0x560c0abf4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 0x560c0abf4020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98e00e0800 tx=0x7f98e0021530 comp rx=0 tx=0).stop 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 -- 172.21.15.16:0/3418913310 shutdown_connections 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7820 0x560c0abf4020 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abe6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 --2- 172.21.15.16:0/3418913310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7410 0x560c0abf1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 -- 172.21.15.16:0/3418913310 >> 172.21.15.16:0/3418913310 conn(0x560c0aac8fc0 msgr2=0x560c0abe2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f98f1e00700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 0x7f98d40b4fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98e40060b0 tx=0x7f98e400b040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.941+0000 7f99028e4b80 1 -- 172.21.15.16:0/3418913310 shutdown_connections 2024-05-27T07:05:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.942+0000 7f99028e4b80 1 -- 172.21.15.16:0/3418913310 wait complete. 2024-05-27T07:05:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.943+0000 7f99028e4b80 1 -- start start 2024-05-27T07:05:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.943+0000 7f99028e4b80 1 Processor -- start 2024-05-27T07:05:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.945+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abf1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560c0aa0bfa0 con 0x560c0aa68e40 2024-05-27T07:05:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c0aaa6760 con 0x560c0abe7820 2024-05-27T07:05:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c0abf4750 con 0x560c0abe7410 2024-05-27T07:05:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f99028e4b80 10 client.0 _ll_register_callbacks cb 0x560c0aaafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f98f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f98f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34466/0 (socket says 172.21.15.16:34466) 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.946+0000 7f98f1e00700 1 -- 172.21.15.16:0/1772442131 learned_addr learned my addr 172.21.15.16:0/1772442131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f0a00700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abf1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f1e00700 1 -- 172.21.15.16:0/1772442131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 msgr2=0x560c0abf1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f1e00700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abf1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f1e00700 1 -- 172.21.15.16:0/1772442131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 msgr2=0x560c0abf1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f1e00700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.947+0000 7f98f1e00700 1 -- 172.21.15.16:0/1772442131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0aa558d0 con 0x560c0abe7820 2024-05-27T07:05:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.948+0000 7f98f1e00700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98e4016d80 tx=0x7f98e4009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966d44c0e5c708a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.948+0000 7f98d3400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.948+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4026060 con 0x560c0abe7820 2024-05-27T07:05:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.948+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f98e4007d50 con 0x560c0abe7820 2024-05-27T07:05:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.949+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e00a9f30 con 0x560c0abe7820 2024-05-27T07:05:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.950+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f98e402d050 con 0x560c0abe7820 2024-05-27T07:05:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.950+0000 7f99028e4b80 10 client.35795 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.950+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c0aad6060 con 0x560c0abe7820 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.951+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e00a9f30 con 0x560c0abe7820 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f98e4014070 con 0x560c0abe7820 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 10 client.35795 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 10 client.35795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 20 client.35795 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac01750 0x560c0ac03c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 4 client.35795 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0ab17d90 con 0x560c0ac01750 2024-05-27T07:05:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.952+0000 7f98f1400700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac01750 0x560c0ac03c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.954+0000 7f98d3400700 10 client.35795 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:05:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.957+0000 7f98f1400700 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac01750 0x560c0ac03c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a12c775a5806d1a server_cookie=32a0c71be85d9953 in_seq=0 out_seq=0 2024-05-27T07:05:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.958+0000 7f98d3400700 1 -- 172.21.15.16:0/1772442131 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560c0ab17d90 con 0x560c0ac01750 2024-05-27T07:05:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.958+0000 7f98d3400700 10 client.35795 handle_command_reply: tid=0 2024-05-27T07:05:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.958+0000 7f99028e4b80 1 client.35795 shutdown 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.960+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 msgr2=0x560c0abf1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.960+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98e4016d80 tx=0x7f98e4009040 comp rx=0 tx=0).stop 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 shutdown_connections 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac01750 0x560c0ac03c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1fa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 --2- 172.21.15.16:0/1772442131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abf1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f98d3400700 0 client.35795 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.961+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 >> 172.21.15.16:0/1772442131 conn(0x560c0aac8fc0 msgr2=0x560c0abe1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.962+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 shutdown_connections 2024-05-27T07:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.962+0000 7f99028e4b80 1 -- 172.21.15.16:0/1772442131 wait complete. 2024-05-27T07:05:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.962+0000 7f99028e4b80 20 client.35795 trim_cache size 0 max 16384 2024-05-27T07:05:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:03.999+0000 7f99028e4b80 1 Processor -- start 2024-05-27T07:05:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.000+0000 7f99028e4b80 1 -- start start 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b67d0 con 0x560c0aa68e40 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b8310 con 0x560c0abe7820 2024-05-27T07:05:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f99028e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b6ea0 con 0x560c0abe7410 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.002+0000 7f98f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.003+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.003+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:45742/0 (socket says 172.21.15.16:45742) 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.003+0000 7f98f0a00700 1 -- 172.21.15.16:0/2575885866 learned_addr learned my addr 172.21.15.16:0/2575885866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.003+0000 7f98f1e00700 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 -- 172.21.15.16:0/2575885866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 msgr2=0x560c0abf1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 -- 172.21.15.16:0/2575885866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 msgr2=0x560c0abecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abecd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 -- 172.21.15.16:0/2575885866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0abe4350 con 0x560c0abe7410 2024-05-27T07:05:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.004+0000 7f98f0a00700 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009990 tx=0x7f98e0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb6a673f1f9a244a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.005+0000 7f98de000700 1 -- 172.21.15.16:0/2575885866 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00a9b90 con 0x560c0abe7410 2024-05-27T07:05:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.005+0000 7f98de000700 1 -- 172.21.15.16:0/2575885866 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e00aab20 con 0x560c0abe7410 2024-05-27T07:05:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.005+0000 7f98de000700 1 -- 172.21.15.16:0/2575885866 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00a9d70 con 0x560c0abe7410 2024-05-27T07:05:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.008+0000 7f99028e4b80 1 -- 172.21.15.16:0/2575885866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 msgr2=0x560c0abfd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.008+0000 7f99028e4b80 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009990 tx=0x7f98e0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 -- 172.21.15.16:0/2575885866 shutdown_connections 2024-05-27T07:05:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 --2- 172.21.15.16:0/2575885866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfd800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 -- 172.21.15.16:0/2575885866 >> 172.21.15.16:0/2575885866 conn(0x560c0aac8fc0 msgr2=0x560c0abf1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.009+0000 7f99028e4b80 1 -- 172.21.15.16:0/2575885866 shutdown_connections 2024-05-27T07:05:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.010+0000 7f99028e4b80 1 -- 172.21.15.16:0/2575885866 wait complete. 2024-05-27T07:05:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.011+0000 7f99028e4b80 1 -- start start 2024-05-27T07:05:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.011+0000 7f99028e4b80 1 Processor -- start 2024-05-27T07:05:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.014+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.015+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.015+0000 7f99028e4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.015+0000 7f99028e4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b7c40 con 0x560c0aa68e40 2024-05-27T07:05:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.015+0000 7f99028e4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b7570 con 0x560c0abe7820 2024-05-27T07:05:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.015+0000 7f99028e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b4770 con 0x560c0abe7410 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.016+0000 7f99028e4b80 10 client.0 _ll_register_callbacks cb 0x560c0aaafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.016+0000 7f98f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.016+0000 7f98f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.016+0000 7f98f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.017+0000 7f98f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34496/0 (socket says 172.21.15.16:34496) 2024-05-27T07:05:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.017+0000 7f98f1e00700 1 -- 172.21.15.16:0/344690380 learned_addr learned my addr 172.21.15.16:0/344690380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.019+0000 7f98f1e00700 1 -- 172.21.15.16:0/344690380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 msgr2=0x560c0abfa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.019+0000 7f98f1e00700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.019+0000 7f98f1e00700 1 -- 172.21.15.16:0/344690380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 msgr2=0x560c0abf9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.019+0000 7f98f1e00700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.019+0000 7f98f1e00700 1 -- 172.21.15.16:0/344690380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0abf5800 con 0x560c0abe7820 2024-05-27T07:05:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.020+0000 7f98f1400700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:05:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.021+0000 7f98f1e00700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009a00 tx=0x7f98e4005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47f9ca9720e12513 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.021+0000 7f98d3400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.022+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4035020 con 0x560c0abe7820 2024-05-27T07:05:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.023+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f98e40208c0 con 0x560c0abe7820 2024-05-27T07:05:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.024+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e00aab20 con 0x560c0abe7820 2024-05-27T07:05:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.025+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f98e4020ac0 con 0x560c0abe7820 2024-05-27T07:05:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.025+0000 7f99028e4b80 10 client.35801 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.025+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c0ac6bd50 con 0x560c0abe7820 2024-05-27T07:05:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.026+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e00aab20 con 0x560c0abe7820 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f98e4014040 con 0x560c0abe7820 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 10 client.35801 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 10 client.35801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 20 client.35801 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac6c3e0 0x560c0ac6e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 4 client.35801 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.027+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c0ab11570 con 0x560c0ac6c3e0 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.028+0000 7f98f1400700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac6c3e0 0x560c0ac6e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.029+0000 7f98f1400700 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac6c3e0 0x560c0ac6e800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52c70acac8105b30 server_cookie=8c0c7f20e0acfdbd in_seq=0 out_seq=0 2024-05-27T07:05:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.029+0000 7f98d3400700 10 client.35801 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:05:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.029+0000 7f98d3400700 1 -- 172.21.15.16:0/344690380 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c0ab11570 con 0x560c0ac6c3e0 2024-05-27T07:05:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.029+0000 7f98d3400700 10 client.35801 handle_command_reply: tid=0 2024-05-27T07:05:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.029+0000 7f99028e4b80 1 client.35801 shutdown 2024-05-27T07:05:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.030+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 msgr2=0x560c0abf5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.030+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009a00 tx=0x7f98e4005ce0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 shutdown_connections 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560c0ac6c3e0 0x560c0ac6e800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c0abe7820 0x560c0abf5090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c0aa68e40 0x560c0abf9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 --2- 172.21.15.16:0/344690380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c0abe7410 0x560c0abfa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f98d3400700 0 client.35801 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:05:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 >> 172.21.15.16:0/344690380 conn(0x560c0aac8fc0 msgr2=0x560c0abe2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 shutdown_connections 2024-05-27T07:05:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.032+0000 7f99028e4b80 1 -- 172.21.15.16:0/344690380 wait complete. 2024-05-27T07:05:04.025 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:05:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.033+0000 7f99028e4b80 20 client.35801 trim_cache size 0 max 16384 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 msgr2=0x7f98d40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 0x7f98d40b4fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98e40060b0 tx=0x7f98e400b040 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 msgr2=0x7f98ec10c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098e0 tx=0x7f98e000f510 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 shutdown_connections 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f98d40b2aa0 0x7f98d40b4fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec108530 0x7f98ec10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ec1032b0 0x7f98ec10c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 --2- 172.21.15.16:0/1321055072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec107be0 0x7f98ec10c760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.034+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 >> 172.21.15.16:0/1321055072 conn(0x7f98ec078be0 msgr2=0x7f98ec1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.035+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 shutdown_connections 2024-05-27T07:05:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:04.035+0000 7f98f2800700 1 -- 172.21.15.16:0/1321055072 wait complete. 2024-05-27T07:05:05.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:04 smithi164 ceph-mon[35327]: pgmap v128: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:05.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:04 smithi016 ceph-mon[31415]: pgmap v128: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:05.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:04 smithi069 ceph-mon[35276]: pgmap v128: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 31 MiB/s wr, 505 op/s 2024-05-27T07:05:05.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:05:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:07.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:06 smithi016 ceph-mon[31415]: pgmap v129: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 770 op/s 2024-05-27T07:05:07.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:06 smithi164 ceph-mon[35327]: pgmap v129: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 770 op/s 2024-05-27T07:05:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:06 smithi069 ceph-mon[35276]: pgmap v129: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 MiB/s wr, 770 op/s 2024-05-27T07:05:09.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:08 smithi164 ceph-mon[35327]: pgmap v130: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 521 op/s 2024-05-27T07:05:09.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:08 smithi016 ceph-mon[31415]: pgmap v130: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 521 op/s 2024-05-27T07:05:09.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:08 smithi069 ceph-mon[35276]: pgmap v130: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 521 op/s 2024-05-27T07:05:11.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:10 smithi164 ceph-mon[35327]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-05-27T07:05:11.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:10 smithi016 ceph-mon[31415]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-05-27T07:05:11.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:10 smithi069 ceph-mon[35276]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-05-27T07:05:13.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:12 smithi164 ceph-mon[35327]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:13.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:12 smithi016 ceph-mon[31415]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:13.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:12 smithi069 ceph-mon[35276]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.484+0000 7f870d400700 1 -- 172.21.15.16:0/2250957402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 msgr2=0x7f87081049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.484+0000 7f870d400700 1 --2- 172.21.15.16:0/2250957402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f87081049f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86f40099c0 tx=0x7f86f4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.484+0000 7f870d400700 1 -- 172.21.15.16:0/2250957402 shutdown_connections 2024-05-27T07:05:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.484+0000 7f870d400700 1 --2- 172.21.15.16:0/2250957402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8708109860 0x7f87081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.484+0000 7f870d400700 1 --2- 172.21.15.16:0/2250957402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f87081049f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.485+0000 7f870d400700 1 --2- 172.21.15.16:0/2250957402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f8708109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.485+0000 7f870d400700 1 -- 172.21.15.16:0/2250957402 >> 172.21.15.16:0/2250957402 conn(0x7f8708100030 msgr2=0x7f8708102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.485+0000 7f870d400700 1 -- 172.21.15.16:0/2250957402 shutdown_connections 2024-05-27T07:05:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.486+0000 7f870d400700 1 -- 172.21.15.16:0/2250957402 wait complete. 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.488+0000 7f870d400700 1 Processor -- start 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.490+0000 7f870d400700 1 -- start start 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.493+0000 7f870d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.493+0000 7f870d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f870806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.493+0000 7f870d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8708109860 0x7f870806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.494+0000 7f870d400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8708114390 con 0x7f8708109860 2024-05-27T07:05:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.494+0000 7f870d400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8708114210 con 0x7f8708108f10 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.494+0000 7f870d400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8708114510 con 0x7f87081045e0 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.495+0000 7f8706800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.495+0000 7f8706800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:45772/0 (socket says 172.21.15.16:45772) 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.495+0000 7f8706800700 1 -- 172.21.15.16:0/3286678973 learned_addr learned my addr 172.21.15.16:0/3286678973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8705e00700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f870806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 -- 172.21.15.16:0/3286678973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 msgr2=0x7f870806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f870806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 -- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8708109860 msgr2=0x7f870806ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8708109860 0x7f870806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 -- 172.21.15.16:0/3286678973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8708072f40 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8705e00700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f870806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.496+0000 7f8706800700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4005330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.497+0000 7f86ff400700 1 -- 172.21.15.16:0/3286678973 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4026070 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.497+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87080731d0 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.497+0000 7f86ff400700 1 -- 172.21.15.16:0/3286678973 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f4003c50 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.497+0000 7f86ff400700 1 -- 172.21.15.16:0/3286678973 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f400faa0 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.497+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8708073700 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.505+0000 7f86ff400700 1 -- 172.21.15.16:0/3286678973 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f86f4029070 con 0x7f87081045e0 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.508+0000 7f86ff400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 0x7f86ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.509+0000 7f86ff400700 1 -- 172.21.15.16:0/3286678973 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f86f40e32b0 con 0x7f87081045e0 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.510+0000 7f8705e00700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 0x7f86ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.512+0000 7f8705e00700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 0x7f86ec0b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8708107750 tx=0x7f86f8009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.512+0000 7f871be79b80 1 Processor -- start 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.512+0000 7f871be79b80 1 -- start start 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 0x563e4e2a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 0x563e4e29c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4e1c9e40 con 0x563e4e2a1d30 2024-05-27T07:05:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563e4e1b99e0 con 0x563e4e11ce40 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.513+0000 7f871be79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e4e1bda20 con 0x563e4e29bdb0 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.514+0000 7f8705e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.514+0000 7f8705e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38308/0 (socket says 172.21.15.16:38308) 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.514+0000 7f8705e00700 1 -- 172.21.15.16:0/2395795296 learned_addr learned my addr 172.21.15.16:0/2395795296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.514+0000 7f8707200700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 0x563e4e29c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 0x563e4e2a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 -- 172.21.15.16:0/2395795296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 msgr2=0x563e4e29c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 0x563e4e29c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 -- 172.21.15.16:0/2395795296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 msgr2=0x563e4e29b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8706800700 1 -- 172.21.15.16:0/2395795296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4e182420 con 0x563e4e11ce40 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.515+0000 7f8707200700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 0x563e4e29c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.516+0000 7f8706800700 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 0x563e4e2a17f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86f40e0610 tx=0x7f86f40a6220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324345f5743f9052 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.516+0000 7f86f3e00700 1 -- 172.21.15.16:0/2395795296 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40aa930 con 0x563e4e11ce40 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.516+0000 7f86f3e00700 1 -- 172.21.15.16:0/2395795296 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f4003e00 con 0x563e4e11ce40 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.516+0000 7f86f3e00700 1 -- 172.21.15.16:0/2395795296 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40b2dd0 con 0x563e4e11ce40 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.517+0000 7f871be79b80 1 -- 172.21.15.16:0/2395795296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 msgr2=0x563e4e2a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.517+0000 7f871be79b80 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 0x563e4e2a17f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86f40e0610 tx=0x7f86f40a6220 comp rx=0 tx=0).stop 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.518+0000 7f871be79b80 1 -- 172.21.15.16:0/2395795296 shutdown_connections 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.518+0000 7f871be79b80 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e29bdb0 0x563e4e29c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.518+0000 7f871be79b80 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e11ce40 0x563e4e2a17f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.518+0000 7f871be79b80 1 --2- 172.21.15.16:0/2395795296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.518+0000 7f871be79b80 1 -- 172.21.15.16:0/2395795296 >> 172.21.15.16:0/2395795296 conn(0x563e4e17cfc0 msgr2=0x563e4e2968d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.521+0000 7f871be79b80 1 -- 172.21.15.16:0/2395795296 shutdown_connections 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.521+0000 7f871be79b80 1 -- 172.21.15.16:0/2395795296 wait complete. 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.522+0000 7f871be79b80 1 -- start start 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.522+0000 7f871be79b80 1 Processor -- start 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.525+0000 7f871be79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.525+0000 7f871be79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.525+0000 7f8706800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.525+0000 7f871be79b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.525+0000 7f8706800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:45800/0 (socket says 172.21.15.16:45800) 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f8706800700 1 -- 172.21.15.16:0/602603312 learned_addr learned my addr 172.21.15.16:0/602603312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4e0bffa0 con 0x563e4e29bdb0 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563e4e15a760 con 0x563e4e2a1d30 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e4e2b0dd0 con 0x563e4e11ce40 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f871be79b80 10 client.0 _ll_register_callbacks cb 0x563e4e163c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.526+0000 7f8705e00700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.527+0000 7f8706800700 1 -- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 msgr2=0x563e4e2a6000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.527+0000 7f8706800700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2a6000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.527+0000 7f8706800700 1 -- 172.21.15.16:0/602603312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 msgr2=0x563e4e2a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.527+0000 7f8706800700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.527+0000 7f8706800700 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4e1098d0 con 0x563e4e11ce40 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.528+0000 7f8706800700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86f40e0670 tx=0x7f86f4009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d4b8582bd4a32e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.528+0000 7f86f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:05:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.529+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40ae5e0 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.529+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f86f40a6820 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.530+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4003e00 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.532+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f86f40aa930 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.532+0000 7f871be79b80 10 client.25756 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.532+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e4e18a060 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.532+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f4003e00 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f86f40c6b50 con 0x563e4e11ce40 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 10 client.25756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 20 client.25756 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e2bdb40 0x563e4e2bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 4 client.25756 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.533+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e4e1cbd90 con 0x563e4e2bdb40 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.534+0000 7f8705e00700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e2bdb40 0x563e4e2bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.535+0000 7f8705e00700 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e2bdb40 0x563e4e2bff60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6344f700fc8a22c server_cookie=9262b88454f78580 in_seq=0 out_seq=0 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.535+0000 7f86f0c00700 10 client.25756 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.537+0000 7f86f0c00700 1 -- 172.21.15.16:0/602603312 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563e4e1cbd90 con 0x563e4e2bdb40 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.537+0000 7f86f0c00700 10 client.25756 handle_command_reply: tid=0 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.537+0000 7f871be79b80 1 client.25756 shutdown 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.539+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 msgr2=0x563e4e2a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.539+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86f40e0670 tx=0x7f86f4009d10 comp rx=0 tx=0).stop 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.539+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 shutdown_connections 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.539+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e2bdb40 0x563e4e2bff60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.540+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.540+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e11ce40 0x563e4e2a5580 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86f40e0670 tx=0x7f86f4009d10 comp rx=0 tx=0).stop 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.540+0000 7f871be79b80 1 --2- 172.21.15.16:0/602603312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.540+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 >> 172.21.15.16:0/602603312 conn(0x563e4e17cfc0 msgr2=0x563e4e296580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.543+0000 7f86f0c00700 0 client.25756 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.543+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 shutdown_connections 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.543+0000 7f871be79b80 1 -- 172.21.15.16:0/602603312 wait complete. 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.544+0000 7f871be79b80 20 client.25756 trim_cache size 0 max 16384 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.577+0000 7f871be79b80 1 Processor -- start 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.577+0000 7f871be79b80 1 -- start start 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.577+0000 7f871be79b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e29bdb0 0x563e4e29b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.577+0000 7f871be79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f871be79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2a5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f871be79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b6b00 con 0x563e4e2a1d30 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f871be79b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b8640 con 0x563e4e29bdb0 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f871be79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b71d0 con 0x563e4e296b60 2024-05-27T07:05:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f8705e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f8705e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38338/0 (socket says 172.21.15.16:38338) 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f8705e00700 1 -- 172.21.15.16:0/660441898 learned_addr learned my addr 172.21.15.16:0/660441898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f8706800700 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e29bdb0 0x563e4e29b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.578+0000 7f8707200700 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.579+0000 7f8707200700 1 -- 172.21.15.16:0/660441898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e29bdb0 msgr2=0x563e4e29b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.579+0000 7f8707200700 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e29bdb0 0x563e4e29b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.579+0000 7f8707200700 1 -- 172.21.15.16:0/660441898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 msgr2=0x563e4e29f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.579+0000 7f8707200700 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.579+0000 7f8707200700 1 -- 172.21.15.16:0/660441898 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4e29ca30 con 0x563e4e296b60 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.580+0000 7f8707200700 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2a5360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87000093a0 tx=0x7f870000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672c1be16508330 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.581+0000 7f86f3400700 1 -- 172.21.15.16:0/660441898 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700003a00 con 0x563e4e296b60 2024-05-27T07:05:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.581+0000 7f86f3400700 1 -- 172.21.15.16:0/660441898 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f870001a070 con 0x563e4e296b60 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.581+0000 7f86f3400700 1 -- 172.21.15.16:0/660441898 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700015400 con 0x563e4e296b60 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.582+0000 7f871be79b80 1 -- 172.21.15.16:0/660441898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 msgr2=0x563e4e2a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.582+0000 7f871be79b80 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2a5360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87000093a0 tx=0x7f870000b040 comp rx=0 tx=0).stop 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.582+0000 7f871be79b80 1 -- 172.21.15.16:0/660441898 shutdown_connections 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2a5360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e29bdb0 0x563e4e29b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 --2- 172.21.15.16:0/660441898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e2a1d30 0x563e4e29f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 -- 172.21.15.16:0/660441898 >> 172.21.15.16:0/660441898 conn(0x563e4e17cfc0 msgr2=0x563e4e29f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 -- 172.21.15.16:0/660441898 shutdown_connections 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 -- 172.21.15.16:0/660441898 wait complete. 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.583+0000 7f871be79b80 1 -- start start 2024-05-27T07:05:15.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.584+0000 7f871be79b80 1 Processor -- start 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2c1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b7f70 con 0x563e4e29bdb0 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b78a0 con 0x563e4e2a1d30 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b4aa0 con 0x563e4e296b60 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.585+0000 7f871be79b80 10 client.0 _ll_register_callbacks cb 0x563e4e163c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.586+0000 7f8707200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.586+0000 7f8707200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34582/0 (socket says 172.21.15.16:34582) 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.586+0000 7f8707200700 1 -- 172.21.15.16:0/1279379146 learned_addr learned my addr 172.21.15.16:0/1279379146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.586+0000 7f8706800700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8705e00700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8707200700 1 -- 172.21.15.16:0/1279379146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 msgr2=0x563e4e2c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8707200700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2c0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8707200700 1 -- 172.21.15.16:0/1279379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 msgr2=0x563e4e2c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8707200700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.587+0000 7f8707200700 1 -- 172.21.15.16:0/1279379146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4e3273b0 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.588+0000 7f8705e00700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.588+0000 7f86f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.588+0000 7f8707200700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8700009960 tx=0x7f8700003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2484089b1e92768f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.589+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700004520 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.589+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8700028470 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.589+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f870001a070 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.590+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f87000286b0 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.590+0000 7f871be79b80 10 client.35813 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.590+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e4e328590 con 0x563e4e2a1d30 2024-05-27T07:05:15.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.590+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f870001a070 con 0x563e4e2a1d30 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f870001e4b0 con 0x563e4e2a1d30 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 10 client.35813 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 10 client.35813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 20 client.35813 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e328b20 0x563e4e32af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 4 client.35813 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.591+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e4e1c5570 con 0x563e4e328b20 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.592+0000 7f8706800700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e328b20 0x563e4e32af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.593+0000 7f8706800700 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e328b20 0x563e4e32af40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=203f0874138c71e8 server_cookie=17de47f5a514ce4 in_seq=0 out_seq=0 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.593+0000 7f86f0c00700 10 client.35813 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.594+0000 7f86f0c00700 1 -- 172.21.15.16:0/1279379146 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e4e1c5570 con 0x563e4e328b20 2024-05-27T07:05:15.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.594+0000 7f86f0c00700 10 client.35813 handle_command_reply: tid=0 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.594+0000 7f871be79b80 1 client.35813 shutdown 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 msgr2=0x563e4e2c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8700009960 tx=0x7f8700003ca0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 shutdown_connections 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x563e4e328b20 0x563e4e32af40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e4e2a1d30 0x563e4e2c1770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e4e296b60 0x563e4e2c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f86f0c00700 0 client.35813 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 --2- 172.21.15.16:0/1279379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4e29bdb0 0x563e4e2c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 >> 172.21.15.16:0/1279379146 conn(0x563e4e17cfc0 msgr2=0x563e4e2b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.595+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 shutdown_connections 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.596+0000 7f871be79b80 1 -- 172.21.15.16:0/1279379146 wait complete. 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.596+0000 7f871be79b80 20 client.35813 trim_cache size 0 max 16384 2024-05-27T07:05:15.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 msgr2=0x7f86ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 0x7f86ec0b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8708107750 tx=0x7f86f8009260 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 msgr2=0x7f8708075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4005330 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 shutdown_connections 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f86ec0b2a70 0x7f86ec0b4f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8708109860 0x7f870806ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87081045e0 0x7f8708075410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 --2- 172.21.15.16:0/3286678973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708108f10 0x7f870806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.597+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 >> 172.21.15.16:0/3286678973 conn(0x7f8708100030 msgr2=0x7f8708102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.598+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 shutdown_connections 2024-05-27T07:05:15.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:10.598+0000 7f870d400700 1 -- 172.21.15.16:0/3286678973 wait complete. 2024-05-27T07:05:15.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:14 smithi164 ceph-mon[35327]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:15.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:14 smithi016 ceph-mon[31415]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:15.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:14 smithi069 ceph-mon[35276]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:05:16.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:05:17.002 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:17.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:16 smithi016 ceph-mon[31415]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 46 MiB/s wr, 767 op/s 2024-05-27T07:05:17.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:16 smithi164 ceph-mon[35327]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 46 MiB/s wr, 767 op/s 2024-05-27T07:05:17.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:16 smithi069 ceph-mon[35276]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 46 MiB/s wr, 767 op/s 2024-05-27T07:05:19.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:19 smithi164 ceph-mon[35327]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 31 MiB/s wr, 502 op/s 2024-05-27T07:05:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:19 smithi016 ceph-mon[31415]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 31 MiB/s wr, 502 op/s 2024-05-27T07:05:19.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:19 smithi069 ceph-mon[35276]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 31 MiB/s wr, 502 op/s 2024-05-27T07:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.244+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2657644688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 msgr2=0x7f8f74110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.244+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2657644688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f74110720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098f0 tx=0x7f8f60009c00 comp rx=0 tx=0).stop 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2657644688 shutdown_connections 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2657644688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f74110c60 0x7f8f74113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2657644688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f741030b0 0x7f8f741034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2657644688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f74110720 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2657644688 >> 172.21.15.16:0/2657644688 conn(0x7f8f740fedb0 msgr2=0x7f8f741011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2657644688 shutdown_connections 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.245+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2657644688 wait complete. 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.247+0000 7f8f7ac00700 1 Processor -- start 2024-05-27T07:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.247+0000 7f8f7ac00700 1 -- start start 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f7410a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74110c60 0x7f8f7410ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74116590 con 0x7f8f74110c60 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74116410 con 0x7f8f741030b0 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f7ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74116710 con 0x7f8f74103a00 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f73400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f7410a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53974/0 (socket says 172.21.15.16:53974) 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f73400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f7410a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47250/0 (socket says 172.21.15.16:47250) 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f73e00700 1 -- 172.21.15.16:0/2312633906 learned_addr learned my addr 172.21.15.16:0/2312633906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.248+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74110c60 0x7f8f7410ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 -- 172.21.15.16:0/2312633906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 msgr2=0x7f8f7410a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f7410a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 -- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74110c60 msgr2=0x7f8f7410ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74110c60 0x7f8f7410ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 -- 172.21.15.16:0/2312633906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f74110760 con 0x7f8f741030b0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.249+0000 7f8f73e00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d980 tx=0x7f8f6400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.250+0000 7f8f70c00700 1 -- 172.21.15.16:0/2312633906 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64016070 con 0x7f8f741030b0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.250+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f74110a50 con 0x7f8f741030b0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.250+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f74105760 con 0x7f8f741030b0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.250+0000 7f8f70c00700 1 -- 172.21.15.16:0/2312633906 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f64004830 con 0x7f8f741030b0 2024-05-27T07:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.251+0000 7f8f70c00700 1 -- 172.21.15.16:0/2312633906 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640054c0 con 0x7f8f741030b0 2024-05-27T07:05:21.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.259+0000 7f8f70c00700 1 -- 172.21.15.16:0/2312633906 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8f64004220 con 0x7f8f741030b0 2024-05-27T07:05:21.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.260+0000 7f8f89646b80 1 Processor -- start 2024-05-27T07:05:21.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.260+0000 7f8f89646b80 1 -- start start 2024-05-27T07:05:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.260+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.260+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74e1d30 0x5558e74db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.261+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.261+0000 7f8f89646b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558e7409e40 con 0x5558e74e1d30 2024-05-27T07:05:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.261+0000 7f8f89646b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5558e73f99e0 con 0x5558e74dbdb0 2024-05-27T07:05:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.261+0000 7f8f89646b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558e73fda20 con 0x5558e735ce40 2024-05-27T07:05:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47256/0 (socket says 172.21.15.16:47256) 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 -- 172.21.15.16:0/2905738705 learned_addr learned my addr 172.21.15.16:0/2905738705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f73400700 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 -- 172.21.15.16:0/2905738705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 msgr2=0x5558e74dc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74dc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 -- 172.21.15.16:0/2905738705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74e1d30 msgr2=0x5558e74db870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74e1d30 0x5558e74db870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.262+0000 7f8f78c00700 1 -- 172.21.15.16:0/2905738705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e73c2420 con 0x5558e735ce40 2024-05-27T07:05:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.263+0000 7f8f70c00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 0x7f8f580b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.263+0000 7f8f70c00700 1 -- 172.21.15.16:0/2312633906 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8f640e2610 con 0x7f8f741030b0 2024-05-27T07:05:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.263+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0093a0 tx=0x7f8f6c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b87631f3cf7524 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.264+0000 7f8f69600700 1 -- 172.21.15.16:0/2905738705 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c003c00 con 0x5558e735ce40 2024-05-27T07:05:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.265+0000 7f8f69600700 1 -- 172.21.15.16:0/2905738705 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f6c003da0 con 0x5558e735ce40 2024-05-27T07:05:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.265+0000 7f8f69600700 1 -- 172.21.15.16:0/2905738705 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0178e0 con 0x5558e735ce40 2024-05-27T07:05:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.266+0000 7f8f89646b80 1 -- 172.21.15.16:0/2905738705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 msgr2=0x5558e74e17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.266+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0093a0 tx=0x7f8f6c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.267+0000 7f8f89646b80 1 -- 172.21.15.16:0/2905738705 shutdown_connections 2024-05-27T07:05:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.267+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74dc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.267+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e17f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.267+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2905738705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74e1d30 0x5558e74db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.267+0000 7f8f89646b80 1 -- 172.21.15.16:0/2905738705 >> 172.21.15.16:0/2905738705 conn(0x5558e73bcfc0 msgr2=0x5558e74d68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.270+0000 7f8f73e00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 0x7f8f580b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.270+0000 7f8f89646b80 1 -- 172.21.15.16:0/2905738705 shutdown_connections 2024-05-27T07:05:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.270+0000 7f8f89646b80 1 -- 172.21.15.16:0/2905738705 wait complete. 2024-05-27T07:05:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.271+0000 7f8f89646b80 1 -- start start 2024-05-27T07:05:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.271+0000 7f8f89646b80 1 Processor -- start 2024-05-27T07:05:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.272+0000 7f8f73e00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 0x7f8f580b4fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d950 tx=0x7f8f64007640 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.273+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.273+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.273+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47266/0 (socket says 172.21.15.16:47266) 2024-05-27T07:05:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f78c00700 1 -- 172.21.15.16:0/4021102310 learned_addr learned my addr 172.21.15.16:0/4021102310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f73400700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558e72fffa0 con 0x5558e74dbdb0 2024-05-27T07:05:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5558e739a760 con 0x5558e74e1d30 2024-05-27T07:05:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.274+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558e74f9810 con 0x5558e735ce40 2024-05-27T07:05:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.275+0000 7f8f89646b80 10 client.0 _ll_register_callbacks cb 0x5558e73a3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.276+0000 7f8f78c00700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.276+0000 7f8f78c00700 1 -- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 msgr2=0x5558e74e76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.276+0000 7f8f78c00700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74e76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.276+0000 7f8f78c00700 1 -- 172.21.15.16:0/4021102310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 msgr2=0x5558e74e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.276+0000 7f8f78c00700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.277+0000 7f8f78c00700 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e73498d0 con 0x5558e735ce40 2024-05-27T07:05:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.277+0000 7f8f73400700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:05:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.278+0000 7f8f78c00700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000f80 tx=0x7f8f6c017460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e8bc44ab9ecaaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.278+0000 7f8f5e000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:05:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017500 con 0x5558e735ce40 2024-05-27T07:05:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c003da0 con 0x5558e735ce40 2024-05-27T07:05:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8f6c005040 con 0x5558e735ce40 2024-05-27T07:05:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f6c005240 con 0x5558e735ce40 2024-05-27T07:05:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f89646b80 10 client.25774 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:21.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.279+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5558e73ca060 con 0x5558e735ce40 2024-05-27T07:05:21.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.280+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f6c003da0 con 0x5558e735ce40 2024-05-27T07:05:21.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.280+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8f6c0237a0 con 0x5558e735ce40 2024-05-27T07:05:21.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 20 client.25774 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e75066d0 0x5558e7508af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 4 client.25774 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.281+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558e740bd90 con 0x5558e75066d0 2024-05-27T07:05:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.282+0000 7f8f73400700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e75066d0 0x5558e7508af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.284+0000 7f8f5e000700 10 client.25774 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.284+0000 7f8f73400700 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e75066d0 0x5558e7508af0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506335f2014bd44 server_cookie=46ae5642248a042d in_seq=0 out_seq=0 2024-05-27T07:05:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.285+0000 7f8f5e000700 1 -- 172.21.15.16:0/4021102310 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558e740bd90 con 0x5558e75066d0 2024-05-27T07:05:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.285+0000 7f8f5e000700 10 client.25774 handle_command_reply: tid=0 2024-05-27T07:05:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.285+0000 7f8f89646b80 1 client.25774 shutdown 2024-05-27T07:05:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.286+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 msgr2=0x5558e74e6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.286+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000f80 tx=0x7f8f6c017460 comp rx=0 tx=0).stop 2024-05-27T07:05:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 shutdown_connections 2024-05-27T07:05:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e75066d0 0x5558e7508af0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74e76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e74e6c60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 --2- 172.21.15.16:0/4021102310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.287+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 >> 172.21.15.16:0/4021102310 conn(0x5558e73bcfc0 msgr2=0x5558e74d5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.288+0000 7f8f5e000700 0 client.25774 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.288+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 shutdown_connections 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.288+0000 7f8f89646b80 1 -- 172.21.15.16:0/4021102310 wait complete. 2024-05-27T07:05:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.288+0000 7f8f89646b80 20 client.25774 trim_cache size 0 max 16384 2024-05-27T07:05:21.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.333+0000 7f8f89646b80 1 Processor -- start 2024-05-27T07:05:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.334+0000 7f8f89646b80 1 -- start start 2024-05-27T07:05:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.334+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.334+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e74e1d30 0x5558e74e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f89646b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5400ed20 con 0x5558e735ce40 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f89646b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54010860 con 0x5558e74dbdb0 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43974/0 (socket says 172.21.15.16:43974) 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f89646b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5400f3f0 con 0x5558e74e1d30 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.335+0000 7f8f78c00700 1 -- 172.21.15.16:0/2609372678 learned_addr learned my addr 172.21.15.16:0/2609372678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.336+0000 7f8f73400700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.336+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e74e1d30 0x5558e74e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f73400700 1 -- 172.21.15.16:0/2609372678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e74e1d30 msgr2=0x5558e74e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f73400700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e74e1d30 0x5558e74e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f73400700 1 -- 172.21.15.16:0/2609372678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 msgr2=0x5558e74db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f73400700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f73400700 1 -- 172.21.15.16:0/2609372678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e74e3a80 con 0x5558e74dbdb0 2024-05-27T07:05:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.338+0000 7f8f78c00700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:05:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.339+0000 7f8f73400700 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74e3150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbee15b2e7f050d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.340+0000 7f8f68c00700 1 -- 172.21.15.16:0/2609372678 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000f940 con 0x5558e74dbdb0 2024-05-27T07:05:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.340+0000 7f8f68c00700 1 -- 172.21.15.16:0/2609372678 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f600054d0 con 0x5558e74dbdb0 2024-05-27T07:05:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.340+0000 7f8f68c00700 1 -- 172.21.15.16:0/2609372678 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60003d30 con 0x5558e74dbdb0 2024-05-27T07:05:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.341+0000 7f8f89646b80 1 -- 172.21.15.16:0/2609372678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 msgr2=0x5558e74e3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.341+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74e3150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60009f50 comp rx=0 tx=0).stop 2024-05-27T07:05:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.343+0000 7f8f89646b80 1 -- 172.21.15.16:0/2609372678 shutdown_connections 2024-05-27T07:05:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.343+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e74e1d30 0x5558e74e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.343+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e735ce40 0x5558e74db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.343+0000 7f8f89646b80 1 --2- 172.21.15.16:0/2609372678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74dbdb0 0x5558e74e3150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.343+0000 7f8f89646b80 1 -- 172.21.15.16:0/2609372678 >> 172.21.15.16:0/2609372678 conn(0x5558e73bcfc0 msgr2=0x5558e74d5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.344+0000 7f8f89646b80 1 -- 172.21.15.16:0/2609372678 shutdown_connections 2024-05-27T07:05:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.344+0000 7f8f89646b80 1 -- 172.21.15.16:0/2609372678 wait complete. 2024-05-27T07:05:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.345+0000 7f8f89646b80 1 -- start start 2024-05-27T07:05:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.346+0000 7f8f89646b80 1 Processor -- start 2024-05-27T07:05:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.346+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.346+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f89646b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f89646b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54010190 con 0x5558e74dbdb0 2024-05-27T07:05:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f89646b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5400fac0 con 0x5558e74e1d30 2024-05-27T07:05:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f89646b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5400ccc0 con 0x5558e735ce40 2024-05-27T07:05:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f89646b80 10 client.0 _ll_register_callbacks cb 0x5558e73a3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f73400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f78c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47296/0 (socket says 172.21.15.16:47296) 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f78c00700 1 -- 172.21.15.16:0/3287912293 learned_addr learned my addr 172.21.15.16:0/3287912293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.347+0000 7f8f78c00700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f73400700 1 -- 172.21.15.16:0/3287912293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 msgr2=0x5558e750e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f73400700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f73400700 1 -- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 msgr2=0x5558e74ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f73400700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74ff040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f73400700 1 -- 172.21.15.16:0/3287912293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e74ff600 con 0x5558e74dbdb0 2024-05-27T07:05:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.348+0000 7f8f78c00700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:05:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.349+0000 7f8f73400700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74feb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f7838dc3e023c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.349+0000 7f8f5e000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:05:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.350+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600052e0 con 0x5558e74dbdb0 2024-05-27T07:05:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.351+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8f6000f740 con 0x5558e74dbdb0 2024-05-27T07:05:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.351+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f600054d0 con 0x5558e74dbdb0 2024-05-27T07:05:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.352+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f6000f940 con 0x5558e74dbdb0 2024-05-27T07:05:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.352+0000 7f8f89646b80 10 client.16107 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.352+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5558e75700f0 con 0x5558e74dbdb0 2024-05-27T07:05:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.354+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f600054d0 con 0x5558e74dbdb0 2024-05-27T07:05:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.354+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8f6002d070 con 0x5558e74dbdb0 2024-05-27T07:05:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 20 client.16107 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e7570660 0x5558e7572a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 4 client.16107 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:05:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.355+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558e7405570 con 0x5558e7570660 2024-05-27T07:05:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.357+0000 7f8f78c00700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e7570660 0x5558e7572a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.358+0000 7f8f5e000700 10 client.16107 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.359+0000 7f8f78c00700 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e7570660 0x5558e7572a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c160322c1fa6e1 server_cookie=ebc0fd016744aae5 in_seq=0 out_seq=0 2024-05-27T07:05:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.360+0000 7f8f5e000700 1 -- 172.21.15.16:0/3287912293 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558e7405570 con 0x5558e7570660 2024-05-27T07:05:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.360+0000 7f8f5e000700 10 client.16107 handle_command_reply: tid=0 2024-05-27T07:05:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.360+0000 7f8f89646b80 1 client.16107 shutdown 2024-05-27T07:05:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.361+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 msgr2=0x5558e74feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.361+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74feb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60005680 comp rx=0 tx=0).stop 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 shutdown_connections 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5558e7570660 0x5558e7572a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558e74e1d30 0x5558e74ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558e735ce40 0x5558e750e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 --2- 172.21.15.16:0/3287912293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e74dbdb0 0x5558e74feb00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 >> 172.21.15.16:0/3287912293 conn(0x5558e73bcfc0 msgr2=0x5558e74d6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.362+0000 7f8f5e000700 0 client.16107 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.364+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 shutdown_connections 2024-05-27T07:05:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.364+0000 7f8f89646b80 1 -- 172.21.15.16:0/3287912293 wait complete. 2024-05-27T07:05:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.364+0000 7f8f89646b80 20 client.16107 trim_cache size 0 max 16384 2024-05-27T07:05:21.357 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:05:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 msgr2=0x7f8f580b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 0x7f8f580b4fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d950 tx=0x7f8f64007640 comp rx=0 tx=0).stop 2024-05-27T07:05:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 msgr2=0x7f8f7410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d980 tx=0x7f8f6400dc90 comp rx=0 tx=0).stop 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 shutdown_connections 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8f580b2aa0 0x7f8f580b4fb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74110c60 0x7f8f7410ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f741030b0 0x7f8f7410a210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 --2- 172.21.15.16:0/2312633906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f74103a00 0x7f8f7410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.366+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 >> 172.21.15.16:0/2312633906 conn(0x7f8f740fedb0 msgr2=0x7f8f741128a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.367+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 shutdown_connections 2024-05-27T07:05:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:21.367+0000 7f8f7ac00700 1 -- 172.21.15.16:0/2312633906 wait complete. 2024-05-27T07:05:21.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:21 smithi164 ceph-mon[35327]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:21.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:21 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:05:21.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:21 smithi016 ceph-mon[31415]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:21.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:21 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:05:21.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:21 smithi069 ceph-mon[35276]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:21.869 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:21 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:05:23.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:05:23.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:05:23.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:23 smithi164 ceph-mon[35327]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-05-27T07:05:23.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:23 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:23.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:23 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:23.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:23 smithi016 ceph-mon[31415]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-05-27T07:05:23.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:23 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:23.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:23 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:23 smithi069 ceph-mon[35276]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-05-27T07:05:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:23 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:23.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:23 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:25.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:25 smithi164 ceph-mon[35327]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 32 MiB/s wr, 522 op/s 2024-05-27T07:05:25.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:25 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:05:25.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:25 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:05:25.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:25 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:25.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:25 smithi016 ceph-mon[31415]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 32 MiB/s wr, 522 op/s 2024-05-27T07:05:25.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:25 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:05:25.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:25 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:05:25.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:25 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:25.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:25 smithi069 ceph-mon[35276]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 32 MiB/s wr, 522 op/s 2024-05-27T07:05:25.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:25 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:05:25.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:25 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:05:25.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:25 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:05:27.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:27 smithi164 ceph-mon[35327]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 768 op/s 2024-05-27T07:05:27.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:27 smithi016 ceph-mon[31415]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 768 op/s 2024-05-27T07:05:27.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:27 smithi069 ceph-mon[35276]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 768 op/s 2024-05-27T07:05:29.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:29 smithi016 ceph-mon[31415]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 32 MiB/s wr, 512 op/s 2024-05-27T07:05:29.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:29 smithi164 ceph-mon[35327]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 32 MiB/s wr, 512 op/s 2024-05-27T07:05:29.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:29 smithi069 ceph-mon[35276]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 32 MiB/s wr, 512 op/s 2024-05-27T07:05:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 -- 172.21.15.16:0/2084905634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660108690 msgr2=0x7f5660108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 --2- 172.21.15.16:0/2084905634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660108690 0x7f5660108b60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f564c009990 tx=0x7f564c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 -- 172.21.15.16:0/2084905634 shutdown_connections 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 --2- 172.21.15.16:0/2084905634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660108690 0x7f5660108b60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 --2- 172.21.15.16:0/2084905634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f5660103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 --2- 172.21.15.16:0/2084905634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660103b90 0x7f56601080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 -- 172.21.15.16:0/2084905634 >> 172.21.15.16:0/2084905634 conn(0x7f56600fedb0 msgr2=0x7f56601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.810+0000 7f5667e00700 1 -- 172.21.15.16:0/2084905634 shutdown_connections 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.811+0000 7f5667e00700 1 -- 172.21.15.16:0/2084905634 wait complete. 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.812+0000 7f5667e00700 1 Processor -- start 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.812+0000 7f5667e00700 1 -- start start 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 0x7f56601a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660108690 0x7f56601a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f566011a910 con 0x7f5660108690 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f566011a790 con 0x7f5660103240 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5667e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f566011aa90 con 0x7f5660103b90 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5665400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5665400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37678/0 (socket says 172.21.15.16:37678) 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.813+0000 7f5665400700 1 -- 172.21.15.16:0/2495110854 learned_addr learned my addr 172.21.15.16:0/2495110854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.814+0000 7f5665e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660108690 0x7f56601a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.814+0000 7f5664a00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 0x7f56601a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.815+0000 7f5664a00700 1 -- 172.21.15.16:0/2495110854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 msgr2=0x7f56601a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.815+0000 7f5664a00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.815+0000 7f5664a00700 1 -- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660108690 msgr2=0x7f56601a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.815+0000 7f5664a00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660108690 0x7f56601a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.815+0000 7f5664a00700 1 -- 172.21.15.16:0/2495110854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601a6190 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.816+0000 7f5665400700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.817+0000 7f5664a00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 0x7f56601a1860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f565800ef10 tx=0x7f565800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.818+0000 7f565e000700 1 -- 172.21.15.16:0/2495110854 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800ca70 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.818+0000 7f565e000700 1 -- 172.21.15.16:0/2495110854 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5658019070 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.818+0000 7f565e000700 1 -- 172.21.15.16:0/2495110854 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580146e0 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.818+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601a6420 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.819+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a6910 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.826+0000 7f565e000700 1 -- 172.21.15.16:0/2495110854 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5658012050 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.827+0000 7f565e000700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 0x7f56440b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.828+0000 7f565e000700 1 -- 172.21.15.16:0/2495110854 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f56580e37d0 con 0x7f5660103b90 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.829+0000 7f5665400700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 0x7f56440b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.832+0000 7f5665400700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 0x7f56440b5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5650011550 tx=0x7f5650009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.832+0000 7f5676978b80 1 Processor -- start 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.832+0000 7f5676978b80 1 -- start start 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 0x55d041039870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d041034240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d040f61ec0 con 0x55d041033e30 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d040f51a60 con 0x55d040eb4ec0 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.833+0000 7f5676978b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d040f55aa0 con 0x55d041039db0 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.834+0000 7f5664a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.834+0000 7f5664a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48200/0 (socket says 172.21.15.16:48200) 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.834+0000 7f5664a00700 1 -- 172.21.15.16:0/2943435118 learned_addr learned my addr 172.21.15.16:0/2943435118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.834+0000 7f5665400700 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 0x55d041039870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.835+0000 7f5665e00700 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d041034240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.835+0000 7f5665400700 1 -- 172.21.15.16:0/2943435118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 msgr2=0x55d0410338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.835+0000 7f5665400700 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.836+0000 7f5665400700 1 -- 172.21.15.16:0/2943435118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 msgr2=0x55d041034240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.836+0000 7f5665400700 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d041034240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.836+0000 7f5665400700 1 -- 172.21.15.16:0/2943435118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d040f1a4a0 con 0x55d040eb4ec0 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.837+0000 7f5665400700 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 0x55d041039870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5650011ca0 tx=0x7f5650014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763b3e237dede248 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.838+0000 7f5656a00700 1 -- 172.21.15.16:0/2943435118 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650004830 con 0x55d040eb4ec0 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.838+0000 7f5656a00700 1 -- 172.21.15.16:0/2943435118 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5650005020 con 0x55d040eb4ec0 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.838+0000 7f5656a00700 1 -- 172.21.15.16:0/2943435118 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565001c780 con 0x55d040eb4ec0 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 -- 172.21.15.16:0/2943435118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 msgr2=0x55d041039870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 0x55d041039870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5650011ca0 tx=0x7f5650014000 comp rx=0 tx=0).stop 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 -- 172.21.15.16:0/2943435118 shutdown_connections 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d041034240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d040eb4ec0 0x55d041039870 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5650011ca0 tx=0x7f5650014000 comp rx=0 tx=0).stop 2024-05-27T07:05:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 --2- 172.21.15.16:0/2943435118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.840+0000 7f5676978b80 1 -- 172.21.15.16:0/2943435118 >> 172.21.15.16:0/2943435118 conn(0x55d040f15040 msgr2=0x55d04102e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.841+0000 7f5676978b80 1 -- 172.21.15.16:0/2943435118 shutdown_connections 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.841+0000 7f5676978b80 1 -- 172.21.15.16:0/2943435118 wait complete. 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.842+0000 7f5676978b80 1 -- start start 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.842+0000 7f5676978b80 1 Processor -- start 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.844+0000 7f5676978b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.844+0000 7f5676978b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d04103dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.844+0000 7f5665400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.845+0000 7f5665400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40734/0 (socket says 172.21.15.16:40734) 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.845+0000 7f5665400700 1 -- 172.21.15.16:0/1760777812 learned_addr learned my addr 172.21.15.16:0/1760777812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.845+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d04103e120 0x55d041049bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.845+0000 7f5664a00700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d04103dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.846+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d040e58020 con 0x55d041033e30 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.846+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d040ef27e0 con 0x55d041039db0 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.846+0000 7f5665e00700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d04103e120 0x55d041049bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.846+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d041040880 con 0x55d04103e120 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.847+0000 7f5665400700 1 -- 172.21.15.16:0/1760777812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d04103e120 msgr2=0x55d041049bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.847+0000 7f5665400700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d04103e120 0x55d041049bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.847+0000 7f5665400700 1 -- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 msgr2=0x55d04103dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.847+0000 7f5665400700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d04103dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.848+0000 7f5676978b80 10 client.0 _ll_register_callbacks cb 0x55d040efbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.848+0000 7f5665400700 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d040ea1950 con 0x55d041033e30 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.849+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5650005020 con 0x55d041033e30 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.849+0000 7f5665400700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55d041034bb0 tx=0x7f565000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc3420dd9609ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.849+0000 7f564b400700 10 client.16116 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.850+0000 7f564b400700 1 -- 172.21.15.16:0/1760777812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650036050 con 0x55d041033e30 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.851+0000 7f564b400700 1 -- 172.21.15.16:0/1760777812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f56500053d0 con 0x55d041033e30 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.851+0000 7f564b400700 1 -- 172.21.15.16:0/1760777812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5650004a70 con 0x55d041033e30 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.851+0000 7f5676978b80 10 client.16116 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.851+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55d040f220e0 con 0x55d041033e30 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.852+0000 7f564b400700 1 -- 172.21.15.16:0/1760777812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5650004c10 con 0x55d041033e30 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.852+0000 7f5676978b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.852+0000 7f5676978b80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.853+0000 7f5676978b80 20 client.16116 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.853+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d04104d8f0 0x55d04104fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.853+0000 7f5676978b80 4 client.16116 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.853+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d040f63e10 con 0x55d04104d8f0 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.853+0000 7f5664a00700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d04104d8f0 0x55d04104fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.855+0000 7f5664a00700 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d04104d8f0 0x55d04104fd10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4c9dd460ed2486 server_cookie=b4d196d384c69a4d in_seq=0 out_seq=0 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.856+0000 7f564b400700 10 client.16116 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.857+0000 7f564b400700 1 -- 172.21.15.16:0/1760777812 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d040f63e10 con 0x55d04104d8f0 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.857+0000 7f564b400700 10 client.16116 handle_command_reply: tid=0 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.857+0000 7f5676978b80 1 client.16116 shutdown 2024-05-27T07:05:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 msgr2=0x55d04103d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55d041034bb0 tx=0x7f565000ee60 comp rx=0 tx=0).stop 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 shutdown_connections 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d04104d8f0 0x55d04104fd10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d04103e120 0x55d041049bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041033e30 0x55d04103d6a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 --2- 172.21.15.16:0/1760777812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d04103dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 >> 172.21.15.16:0/1760777812 conn(0x55d040f15040 msgr2=0x55d04102e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.858+0000 7f564b400700 0 client.16116 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.859+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 shutdown_connections 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.859+0000 7f5676978b80 1 -- 172.21.15.16:0/1760777812 wait complete. 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.860+0000 7f5676978b80 20 client.16116 trim_cache size 0 max 16384 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.909+0000 7f5676978b80 1 Processor -- start 2024-05-27T07:05:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.909+0000 7f5676978b80 1 -- start start 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d04103e120 0x55d04103b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041049ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b6b80 con 0x55d04103e120 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b86c0 con 0x55d041039db0 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5676978b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b7250 con 0x55d041049ae0 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.910+0000 7f5665400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37708/0 (socket says 172.21.15.16:37708) 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 -- 172.21.15.16:0/798114740 learned_addr learned my addr 172.21.15.16:0/798114740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665e00700 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041049ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 -- 172.21.15.16:0/798114740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 msgr2=0x55d041049ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041049ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 -- 172.21.15.16:0/798114740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d04103e120 msgr2=0x55d04103b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d04103e120 0x55d04103b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.911+0000 7f5665400700 1 -- 172.21.15.16:0/798114740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d041040760 con 0x55d041039db0 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.912+0000 7f5665400700 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f56500264a0 tx=0x7f5650013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80dd7c1e96c01c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.913+0000 7f5656000700 1 -- 172.21.15.16:0/798114740 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650036040 con 0x55d041039db0 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.913+0000 7f5656000700 1 -- 172.21.15.16:0/798114740 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565003f070 con 0x55d041039db0 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.913+0000 7f5656000700 1 -- 172.21.15.16:0/798114740 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650026d40 con 0x55d041039db0 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.914+0000 7f5676978b80 1 -- 172.21.15.16:0/798114740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 msgr2=0x55d0410338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.914+0000 7f5676978b80 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f56500264a0 tx=0x7f5650013070 comp rx=0 tx=0).stop 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 -- 172.21.15.16:0/798114740 shutdown_connections 2024-05-27T07:05:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041049ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d041039db0 0x55d0410338f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 --2- 172.21.15.16:0/798114740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d04103e120 0x55d04103b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 -- 172.21.15.16:0/798114740 >> 172.21.15.16:0/798114740 conn(0x55d040f15040 msgr2=0x55d04102dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 -- 172.21.15.16:0/798114740 shutdown_connections 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.915+0000 7f5676978b80 1 -- 172.21.15.16:0/798114740 wait complete. 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.916+0000 7f5676978b80 1 -- start start 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.916+0000 7f5676978b80 1 Processor -- start 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.918+0000 7f5676978b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d04103e120 0x55d0410533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.918+0000 7f5676978b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.918+0000 7f5676978b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 0x55d0410520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5664a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5664a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48242/0 (socket says 172.21.15.16:48242) 2024-05-27T07:05:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5676978b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b7ff0 con 0x55d041051cc0 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5676978b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b7920 con 0x55d04103e120 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5676978b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56601b4b20 con 0x55d041049ae0 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.919+0000 7f5676978b80 10 client.0 _ll_register_callbacks cb 0x55d040efbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.920+0000 7f5664a00700 1 -- 172.21.15.16:0/2050088039 learned_addr learned my addr 172.21.15.16:0/2050088039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.920+0000 7f5665400700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d04103e120 0x55d0410533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.920+0000 7f5664a00700 1 -- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d04103e120 msgr2=0x55d0410533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5664a00700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d04103e120 0x55d0410533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5665e00700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 0x55d0410520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5664a00700 1 -- 172.21.15.16:0/2050088039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 msgr2=0x55d0410520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5664a00700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 0x55d0410520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.948 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:05:29.948 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:05:29.948 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "373133c7-73a1-4d4b-9ea6-a1b2c39add21", 2024-05-27T07:05:29.949 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:05:29.949 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:05:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5664a00700 1 -- 172.21.15.16:0/2050088039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0410b8240 con 0x55d041049ae0 2024-05-27T07:05:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.921+0000 7f5665e00700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 0x55d0410520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.922+0000 7f564b400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.922+0000 7f5664a00700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56580a7930 tx=0x7f56580aa300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c8896c824e0902 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.923+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580e0fc0 con 0x55d041049ae0 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.923+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565003f070 con 0x55d041049ae0 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.923+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f565800cc10 con 0x55d041049ae0 2024-05-27T07:05:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.924+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56580aea70 con 0x55d041049ae0 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.925+0000 7f5676978b80 10 client.25798 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.925+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0410b9590 con 0x55d041049ae0 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565003f070 con 0x55d041049ae0 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f56580aa9b0 con 0x55d041049ae0 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 20 client.25798 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d0410b9bf0 0x55d0410bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 4 client.25798 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.927+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d040f5d5f0 con 0x55d0410b9bf0 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.928+0000 7f5665400700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d0410b9bf0 0x55d0410bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.930+0000 7f5665400700 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d0410b9bf0 0x55d0410bc010 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97200f71172db039 server_cookie=83aecf9b84c6e811 in_seq=0 out_seq=0 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.930+0000 7f564b400700 10 client.25798 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.932+0000 7f564b400700 1 -- 172.21.15.16:0/2050088039 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d040f5d5f0 con 0x55d0410b9bf0 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.932+0000 7f564b400700 10 client.25798 handle_command_reply: tid=0 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.933+0000 7f5676978b80 1 client.25798 shutdown 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.934+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 msgr2=0x55d041051780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.934+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56580a7930 tx=0x7f56580aa300 comp rx=0 tx=0).stop 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.934+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 shutdown_connections 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55d0410b9bf0 0x55d0410bc010 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d041051cc0 0x55d0410520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d04103e120 0x55d0410533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 --2- 172.21.15.16:0/2050088039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d041049ae0 0x55d041051780 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 >> 172.21.15.16:0/2050088039 conn(0x55d040f15040 msgr2=0x55d041048ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f564b400700 0 client.25798 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 shutdown_connections 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.935+0000 7f5676978b80 1 -- 172.21.15.16:0/2050088039 wait complete. 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.936+0000 7f5676978b80 20 client.25798 trim_cache size 0 max 16384 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.937+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 msgr2=0x7f56440b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.937+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 0x7f56440b5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5650011550 tx=0x7f5650009260 comp rx=0 tx=0).stop 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.937+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 msgr2=0x7f56601a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.937+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 0x7f56601a1860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f565800ef10 tx=0x7f565800c3a0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 shutdown_connections 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f56440b2af0 0x7f56440b5000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660108690 0x7f56601a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5660103240 0x7f56601a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 --2- 172.21.15.16:0/2495110854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5660103b90 0x7f56601a1860 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.938+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 >> 172.21.15.16:0/2495110854 conn(0x7f56600fedb0 msgr2=0x7f5660106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.939+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 shutdown_connections 2024-05-27T07:05:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:29.939+0000 7f5667e00700 1 -- 172.21.15.16:0/2495110854 wait complete. 2024-05-27T07:05:31.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:05:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:31 smithi164 ceph-mon[35327]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:31 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:05:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:31 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:31 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:05:31.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:31 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:31 smithi016 ceph-mon[31415]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:31 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:05:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:31 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:31 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:05:31.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:31 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:31.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:31 smithi069 ceph-mon[35276]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 41 MiB/s wr, 657 op/s 2024-05-27T07:05:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:31 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:05:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:31 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:31 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:05:31.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:31 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:05:33.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:33 smithi164 ceph-mon[35327]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 610 op/s 2024-05-27T07:05:33.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:33 smithi016 ceph-mon[31415]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 610 op/s 2024-05-27T07:05:33.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:33 smithi069 ceph-mon[35276]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 610 op/s 2024-05-27T07:05:34.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:34 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:05:34.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:34 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:05:35.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:34 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:05:35.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:35 smithi016 ceph-mon[31415]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 31 MiB/s wr, 499 op/s 2024-05-27T07:05:35.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:35 smithi069 ceph-mon[35276]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 31 MiB/s wr, 499 op/s 2024-05-27T07:05:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:35 smithi164 ceph-mon[35327]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 31 MiB/s wr, 499 op/s 2024-05-27T07:05:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 -- 172.21.15.16:0/3576003333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc4107de0 msgr2=0x7fccc41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 --2- 172.21.15.16:0/3576003333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc4107de0 0x7fccc41081f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098f0 tx=0x7fccb8009c00 comp rx=0 tx=0).stop 2024-05-27T07:05:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 -- 172.21.15.16:0/3576003333 shutdown_connections 2024-05-27T07:05:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 --2- 172.21.15.16:0/3576003333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4108730 0x7fccc4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 --2- 172.21.15.16:0/3576003333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc41034b0 0x7fccc41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 --2- 172.21.15.16:0/3576003333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc4107de0 0x7fccc41081f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.519+0000 7fcccbe00700 1 -- 172.21.15.16:0/3576003333 >> 172.21.15.16:0/3576003333 conn(0x7fccc40fedb0 msgr2=0x7fccc41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.522+0000 7fcccbe00700 1 -- 172.21.15.16:0/3576003333 shutdown_connections 2024-05-27T07:05:36.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.522+0000 7fcccbe00700 1 -- 172.21.15.16:0/3576003333 wait complete. 2024-05-27T07:05:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.526+0000 7fcccbe00700 1 Processor -- start 2024-05-27T07:05:36.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.526+0000 7fcccbe00700 1 -- start start 2024-05-27T07:05:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 0x7fccc419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4107de0 0x7fccc419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 0x7fccc419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113290 con 0x7fccc4107de0 2024-05-27T07:05:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113110 con 0x7fccc41034b0 2024-05-27T07:05:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.527+0000 7fcccbe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113410 con 0x7fccc4108730 2024-05-27T07:05:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.528+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 0x7fccc419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.529+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 0x7fccc419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59960/0 (socket says 172.21.15.16:59960) 2024-05-27T07:05:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.529+0000 7fccc9e00700 1 -- 172.21.15.16:0/2819088642 learned_addr learned my addr 172.21.15.16:0/2819088642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.530+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 0x7fccc419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.530+0000 7fccc8a00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4107de0 0x7fccc419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 -- 172.21.15.16:0/2819088642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 msgr2=0x7fccc419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 0x7fccc419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 -- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4107de0 msgr2=0x7fccc419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4107de0 0x7fccc419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 -- 172.21.15.16:0/2819088642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc419fc50 con 0x7fccc41034b0 2024-05-27T07:05:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.531+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 0x7fccc419add0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.535+0000 7fccc2000700 1 -- 172.21.15.16:0/2819088642 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb000cbd0 con 0x7fccc41034b0 2024-05-27T07:05:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.535+0000 7fccc2000700 1 -- 172.21.15.16:0/2819088642 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccb000cd70 con 0x7fccc41034b0 2024-05-27T07:05:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.535+0000 7fccc2000700 1 -- 172.21.15.16:0/2819088642 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0018680 con 0x7fccc41034b0 2024-05-27T07:05:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.535+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc419ff40 con 0x7fccc41034b0 2024-05-27T07:05:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.535+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc410a060 con 0x7fccc41034b0 2024-05-27T07:05:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.552+0000 7fccc2000700 1 -- 172.21.15.16:0/2819088642 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fccb0010430 con 0x7fccc41034b0 2024-05-27T07:05:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.553+0000 7fccda8feb80 1 Processor -- start 2024-05-27T07:05:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.553+0000 7fccda8feb80 1 -- start start 2024-05-27T07:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 0x55a3f89a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3f88cee40 con 0x55a3f8821e40 2024-05-27T07:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3f88be9e0 con 0x55a3f890d9f0 2024-05-27T07:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.556+0000 7fccda8feb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a3f88c2a20 con 0x55a3f89bc110 2024-05-27T07:05:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.557+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.557+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:54498/0 (socket says 172.21.15.16:54498) 2024-05-27T07:05:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.557+0000 7fccc9e00700 1 -- 172.21.15.16:0/1311472107 learned_addr learned my addr 172.21.15.16:0/1311472107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.558+0000 7fccc9e00700 1 -- 172.21.15.16:0/1311472107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 msgr2=0x55a3f89be530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.559+0000 7fccc9400700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 0x55a3f89a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc8a00700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc9e00700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89be530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc9e00700 1 -- 172.21.15.16:0/1311472107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 msgr2=0x55a3f89a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc9e00700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 0x55a3f89a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc9e00700 1 -- 172.21.15.16:0/1311472107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f8887420 con 0x55a3f890d9f0 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.560+0000 7fccc9400700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 0x55a3f89a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.561+0000 7fccc9e00700 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c520 tx=0x7fccbc0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c87e9d4ebf2478c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.561+0000 7fccc2000700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 0x7fccac0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.561+0000 7fccb6a00700 1 -- 172.21.15.16:0/1311472107 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc019020 con 0x55a3f890d9f0 2024-05-27T07:05:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.561+0000 7fccc2000700 1 -- 172.21.15.16:0/2819088642 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fccb00e2ad0 con 0x7fccc41034b0 2024-05-27T07:05:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.562+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 0x7fccac0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.563+0000 7fccb6a00700 1 -- 172.21.15.16:0/1311472107 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccbc007a50 con 0x55a3f890d9f0 2024-05-27T07:05:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.563+0000 7fccb6a00700 1 -- 172.21.15.16:0/1311472107 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc015910 con 0x55a3f890d9f0 2024-05-27T07:05:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 -- 172.21.15.16:0/1311472107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 msgr2=0x55a3f89a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c520 tx=0x7fccbc0099e0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 -- 172.21.15.16:0/1311472107 shutdown_connections 2024-05-27T07:05:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89be530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89a03d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1311472107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f8821e40 0x55a3f89a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.565+0000 7fccda8feb80 1 -- 172.21.15.16:0/1311472107 >> 172.21.15.16:0/1311472107 conn(0x55a3f88bb390 msgr2=0x55a3f899c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.566+0000 7fccda8feb80 1 -- 172.21.15.16:0/1311472107 shutdown_connections 2024-05-27T07:05:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.566+0000 7fccda8feb80 1 -- 172.21.15.16:0/1311472107 wait complete. 2024-05-27T07:05:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.567+0000 7fccc9400700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 0x7fccac0b5080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a3f899ce20 tx=0x7fccb000ef90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.567+0000 7fccda8feb80 1 -- start start 2024-05-27T07:05:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.567+0000 7fccda8feb80 1 Processor -- start 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89bc110 0x55a3f89b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3f87c4fa0 con 0x55a3f89bc110 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3f885f760 con 0x55a3f890d9f0 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.568+0000 7fccda8feb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a3f89c73d0 con 0x55a3f8821e40 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.569+0000 7fccda8feb80 10 client.0 _ll_register_callbacks cb 0x55a3f8868c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.569+0000 7fccc8a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.569+0000 7fccc8a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:54514/0 (socket says 172.21.15.16:54514) 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.569+0000 7fccc8a00700 1 -- 172.21.15.16:0/1487600855 learned_addr learned my addr 172.21.15.16:0/1487600855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.570+0000 7fccc8a00700 1 -- 172.21.15.16:0/1487600855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 msgr2=0x55a3f89b9110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.570+0000 7fccc8a00700 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89b9110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.570+0000 7fccc8a00700 1 -- 172.21.15.16:0/1487600855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89bc110 msgr2=0x55a3f89b9b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.570+0000 7fccc8a00700 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89bc110 0x55a3f89b9b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.570+0000 7fccc8a00700 1 -- 172.21.15.16:0/1487600855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f880e8d0 con 0x55a3f890d9f0 2024-05-27T07:05:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.571+0000 7fccc8a00700 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098f0 tx=0x7fccb80174b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5dd6d3020e71c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.571+0000 7fccab400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.572+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8027060 con 0x55a3f890d9f0 2024-05-27T07:05:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.572+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fccb8005040 con 0x55a3f890d9f0 2024-05-27T07:05:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.573+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccbc007a50 con 0x55a3f890d9f0 2024-05-27T07:05:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.575+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccb8005260 con 0x55a3f890d9f0 2024-05-27T07:05:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.575+0000 7fccda8feb80 10 client.35858 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.575+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3f888f060 con 0x55a3f890d9f0 2024-05-27T07:05:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.577+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccbc007a50 con 0x55a3f890d9f0 2024-05-27T07:05:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fccb80175e0 con 0x55a3f890d9f0 2024-05-27T07:05:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 10 client.35858 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 10 client.35858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 20 client.35858 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89d4310 0x55a3f89d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 4 client.35858 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.578+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f88d0d90 con 0x55a3f89d4310 2024-05-27T07:05:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.579+0000 7fccc9e00700 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89d4310 0x55a3f89d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.581+0000 7fccc9e00700 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89d4310 0x55a3f89d6750 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f0d4363c369e1c server_cookie=80d5e339b11c6ad1 in_seq=0 out_seq=0 2024-05-27T07:05:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.582+0000 7fccab400700 10 client.35858 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.583+0000 7fccab400700 1 -- 172.21.15.16:0/1487600855 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a3f88d0d90 con 0x55a3f89d4310 2024-05-27T07:05:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.583+0000 7fccab400700 10 client.35858 handle_command_reply: tid=0 2024-05-27T07:05:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.583+0000 7fccda8feb80 1 client.35858 shutdown 2024-05-27T07:05:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 msgr2=0x55a3f89b9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098f0 tx=0x7fccb80174b0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 shutdown_connections 2024-05-27T07:05:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89d4310 0x55a3f89d6750 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89bc110 0x55a3f89b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 --2- 172.21.15.16:0/1487600855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f890d9f0 0x55a3f89b9650 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098f0 tx=0x7fccb80174b0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccab400700 0 client.35858 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.584+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 >> 172.21.15.16:0/1487600855 conn(0x55a3f88bb390 msgr2=0x55a3f89a0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.586+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 shutdown_connections 2024-05-27T07:05:36.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.588+0000 7fccda8feb80 1 -- 172.21.15.16:0/1487600855 wait complete. 2024-05-27T07:05:36.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.588+0000 7fccda8feb80 20 client.35858 trim_cache size 0 max 16384 2024-05-27T07:05:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.638+0000 7fccda8feb80 1 Processor -- start 2024-05-27T07:05:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.638+0000 7fccda8feb80 1 -- start start 2024-05-27T07:05:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 0x55a3f89b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4073c20 con 0x55a3f89d04f0 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4075760 con 0x55a3f89bc110 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccda8feb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccc40742f0 con 0x55a3f8821e40 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59982/0 (socket says 172.21.15.16:59982) 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.639+0000 7fccc9e00700 1 -- 172.21.15.16:0/3830742939 learned_addr learned my addr 172.21.15.16:0/3830742939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc8a00700 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 0x55a3f89b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc9e00700 1 -- 172.21.15.16:0/3830742939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 msgr2=0x55a3f89b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 0x55a3f89b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc9e00700 1 -- 172.21.15.16:0/3830742939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 msgr2=0x55a3f89bd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc9e00700 1 -- 172.21.15.16:0/3830742939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f89bb9b0 con 0x55a3f8821e40 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.640+0000 7fccc8a00700 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 0x55a3f89b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.641+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00cee0 tx=0x7fccbc0154f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc83dbda37a1cf58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.642+0000 7fccb6000700 1 -- 172.21.15.16:0/3830742939 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc019040 con 0x55a3f8821e40 2024-05-27T07:05:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.642+0000 7fccb6000700 1 -- 172.21.15.16:0/3830742939 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccbc003c00 con 0x55a3f8821e40 2024-05-27T07:05:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.642+0000 7fccb6000700 1 -- 172.21.15.16:0/3830742939 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc015560 con 0x55a3f8821e40 2024-05-27T07:05:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.643+0000 7fccda8feb80 1 -- 172.21.15.16:0/3830742939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 msgr2=0x55a3f89a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.643+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00cee0 tx=0x7fccbc0154f0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 -- 172.21.15.16:0/3830742939 shutdown_connections 2024-05-27T07:05:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89bd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f8821e40 0x55a3f89a0a20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3830742939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f89bc110 0x55a3f89b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 -- 172.21.15.16:0/3830742939 >> 172.21.15.16:0/3830742939 conn(0x55a3f88bb390 msgr2=0x55a3f89b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 -- 172.21.15.16:0/3830742939 shutdown_connections 2024-05-27T07:05:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.645+0000 7fccda8feb80 1 -- 172.21.15.16:0/3830742939 wait complete. 2024-05-27T07:05:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.646+0000 7fccda8feb80 1 -- start start 2024-05-27T07:05:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.646+0000 7fccda8feb80 1 Processor -- start 2024-05-27T07:05:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4075090 con 0x55a3f89d04f0 2024-05-27T07:05:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccc40749c0 con 0x55a3f8821e40 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4071bc0 con 0x55a3f89bc110 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.648+0000 7fccda8feb80 10 client.0 _ll_register_callbacks cb 0x55a3f8868c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.649+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.649+0000 7fccc9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:54532/0 (socket says 172.21.15.16:54532) 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.649+0000 7fccc9e00700 1 -- 172.21.15.16:0/3440654604 learned_addr learned my addr 172.21.15.16:0/3440654604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.651+0000 7fccc9e00700 1 -- 172.21.15.16:0/3440654604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 msgr2=0x55a3f89ccc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.651+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89ccc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.651+0000 7fccc9e00700 1 -- 172.21.15.16:0/3440654604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 msgr2=0x55a3f89c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.651+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.651+0000 7fccc9e00700 1 -- 172.21.15.16:0/3440654604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f89c7be0 con 0x55a3f8821e40 2024-05-27T07:05:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.652+0000 7fccc9e00700 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0156d0 tx=0x7fccbc004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c5c0c7a4d27b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.653+0000 7fccab400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.653+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc019070 con 0x55a3f8821e40 2024-05-27T07:05:36.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.654+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fccbc0075c0 con 0x55a3f8821e40 2024-05-27T07:05:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.655+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccbc003c00 con 0x55a3f8821e40 2024-05-27T07:05:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.656+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccbc019070 con 0x55a3f8821e40 2024-05-27T07:05:36.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.656+0000 7fccda8feb80 10 client.35861 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:36.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.656+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3f89bea70 con 0x55a3f8821e40 2024-05-27T07:05:36.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.657+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccbc015b90 con 0x55a3f8821e40 2024-05-27T07:05:36.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.657+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fccbc03f430 con 0x55a3f8821e40 2024-05-27T07:05:36.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 10 client.35861 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:36.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 10 client.35861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:05:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 20 client.35861 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89beed0 0x55a3f89c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 4 client.35861 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:05:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.658+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fccbc003c00 con 0x55a3f89beed0 2024-05-27T07:05:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.659+0000 7fccc8a00700 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89beed0 0x55a3f89c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:36.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.660+0000 7fccc8a00700 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89beed0 0x55a3f89c12f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a590a0aade1eb74 server_cookie=5beee4e55230ffbd in_seq=0 out_seq=0 2024-05-27T07:05:36.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.660+0000 7fccab400700 10 client.35861 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:36.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.661+0000 7fccab400700 1 -- 172.21.15.16:0/3440654604 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fccbc003c00 con 0x55a3f89beed0 2024-05-27T07:05:36.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.661+0000 7fccab400700 10 client.35861 handle_command_reply: tid=0 2024-05-27T07:05:36.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.661+0000 7fccda8feb80 1 client.35861 shutdown 2024-05-27T07:05:36.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.661+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 msgr2=0x55a3f89cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.661+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0156d0 tx=0x7fccbc004e70 comp rx=0 tx=0).stop 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 shutdown_connections 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55a3f89beed0 0x55a3f89c12f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3f89d04f0 0x55a3f89c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3f8821e40 0x55a3f89cc6f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 --2- 172.21.15.16:0/3440654604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3f89bc110 0x55a3f89ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 >> 172.21.15.16:0/3440654604 conn(0x55a3f88bb390 msgr2=0x55a3f89a1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.662+0000 7fccab400700 0 client.35861 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:05:36.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.663+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 shutdown_connections 2024-05-27T07:05:36.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.664+0000 7fccda8feb80 1 -- 172.21.15.16:0/3440654604 wait complete. 2024-05-27T07:05:36.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.664+0000 7fccda8feb80 20 client.35861 trim_cache size 0 max 16384 2024-05-27T07:05:36.656 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:05:36.656 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:05:36.656 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:05:36.656 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:05:36.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.665+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 msgr2=0x7fccac0b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.665+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 0x7fccac0b5080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a3f899ce20 tx=0x7fccb000ef90 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.665+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 msgr2=0x7fccc419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.665+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 0x7fccc419add0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000ed00 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 shutdown_connections 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fccac0b2b70 0x7fccac0b5080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccc4108730 0x7fccc419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccc41034b0 0x7fccc419add0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 --2- 172.21.15.16:0/2819088642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc4107de0 0x7fccc419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.666+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 >> 172.21.15.16:0/2819088642 conn(0x7fccc40fedb0 msgr2=0x7fccc41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:36.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.667+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 shutdown_connections 2024-05-27T07:05:36.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:36.667+0000 7fcccbe00700 1 -- 172.21.15.16:0/2819088642 wait complete. 2024-05-27T07:05:37.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:05:37.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:05:37.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:37 smithi016 ceph-mon[31415]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 46 MiB/s wr, 740 op/s 2024-05-27T07:05:37.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:37 smithi069 ceph-mon[35276]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 46 MiB/s wr, 740 op/s 2024-05-27T07:05:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:37 smithi164 ceph-mon[35327]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 46 MiB/s wr, 740 op/s 2024-05-27T07:05:38.652 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:39.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:39 smithi016 ceph-mon[31415]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 30 MiB/s wr, 494 op/s 2024-05-27T07:05:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:39 smithi164 ceph-mon[35327]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 30 MiB/s wr, 494 op/s 2024-05-27T07:05:40.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:39 smithi069 ceph-mon[35276]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 30 MiB/s wr, 494 op/s 2024-05-27T07:05:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:41 smithi164 ceph-mon[35327]: pgmap v146: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 38 MiB/s wr, 623 op/s 2024-05-27T07:05:42.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:41 smithi016 ceph-mon[31415]: pgmap v146: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 38 MiB/s wr, 623 op/s 2024-05-27T07:05:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:41 smithi069 ceph-mon[35276]: pgmap v146: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 38 MiB/s wr, 623 op/s 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 -- 172.21.15.16:0/3347794717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e01034b0 msgr2=0x7f92e01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3347794717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e01034b0 0x7f92e01038c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009920 tx=0x7f92d0009c30 comp rx=0 tx=0).stop 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 -- 172.21.15.16:0/3347794717 shutdown_connections 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3347794717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e0108730 0x7f92e0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3347794717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e01034b0 0x7f92e01038c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3347794717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.435+0000 7f92e4c00700 1 -- 172.21.15.16:0/3347794717 >> 172.21.15.16:0/3347794717 conn(0x7f92e00fedb0 msgr2=0x7f92e01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:43.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.436+0000 7f92e4c00700 1 -- 172.21.15.16:0/3347794717 shutdown_connections 2024-05-27T07:05:43.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.436+0000 7f92e4c00700 1 -- 172.21.15.16:0/3347794717 wait complete. 2024-05-27T07:05:43.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.437+0000 7f92e4c00700 1 Processor -- start 2024-05-27T07:05:43.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.438+0000 7f92e4c00700 1 -- start start 2024-05-27T07:05:43.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.438+0000 7f92e4c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:43.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.438+0000 7f92e4c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e0108730 0x7f92e01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:43.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.438+0000 7f92e4c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 0x7f92e01a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:43.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92dde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:43.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92dde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:38440/0 (socket says 172.21.15.16:38440) 2024-05-27T07:05:43.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92dde00700 1 -- 172.21.15.16:0/3204856980 learned_addr learned my addr 172.21.15.16:0/3204856980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01121c0 con 0x7f92e0108730 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0112040 con 0x7f92e01a5cf0 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0112340 con 0x7f92e0107de0 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92de800700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 0x7f92e01a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.439+0000 7f92d7e00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e0108730 0x7f92e01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 -- 172.21.15.16:0/3204856980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 msgr2=0x7f92e01a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 -- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e0108730 msgr2=0x7f92e01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:43.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e0108730 0x7f92e01a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01a6840 con 0x7f92e01a5cf0 2024-05-27T07:05:43.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.440+0000 7f92de800700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 0x7f92e01a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f92d800efc0 tx=0x7f92d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.441+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80092e0 con 0x7f92e01a5cf0 2024-05-27T07:05:43.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.441+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e01a6ad0 con 0x7f92e01a5cf0 2024-05-27T07:05:43.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.441+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e01ae250 con 0x7f92e01a5cf0 2024-05-27T07:05:43.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.441+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d8004bb0 con 0x7f92e01a5cf0 2024-05-27T07:05:43.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.441+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8004020 con 0x7f92e01a5cf0 2024-05-27T07:05:43.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.444+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92bc005330 con 0x7f92e01a5cf0 2024-05-27T07:05:43.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.448+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f92d801b030 con 0x7f92e01a5cf0 2024-05-27T07:05:43.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.451+0000 7f92d6a00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 0x7f92c40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:43.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.452+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f92d80e24a0 con 0x7f92e01a5cf0 2024-05-27T07:05:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.453+0000 7f92dde00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 0x7f92c40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.453+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92d8022020 con 0x7f92e01a5cf0 2024-05-27T07:05:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.455+0000 7f92dde00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 0x7f92c40b4f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000c00 tx=0x7f92d0005e00 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:43.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:43 smithi016 ceph-mon[31415]: pgmap v147: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 37 MiB/s wr, 599 op/s 2024-05-27T07:05:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:43.997+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92bc005620 con 0x7f92e01a5cf0 2024-05-27T07:05:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.001+0000 7f92d6a00700 1 -- 172.21.15.16:0/3204856980 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f92d8010040 con 0x7f92e01a5cf0 2024-05-27T07:05:43.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:05:43.995 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:05:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.005+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 msgr2=0x7f92c40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.005+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 0x7f92c40b4f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000c00 tx=0x7f92d0005e00 comp rx=0 tx=0).stop 2024-05-27T07:05:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.005+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 msgr2=0x7f92e01a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.005+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 0x7f92e01a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f92d800efc0 tx=0x7f92d800c3a0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 shutdown_connections 2024-05-27T07:05:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92c40b2a50 0x7f92c40b4f60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e01a5cf0 0x7f92e01a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92e0107de0 0x7f92e01a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 --2- 172.21.15.16:0/3204856980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e0108730 0x7f92e01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.006+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 >> 172.21.15.16:0/3204856980 conn(0x7f92e00fedb0 msgr2=0x7f92e01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.007+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 shutdown_connections 2024-05-27T07:05:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:44.008+0000 7f92e4c00700 1 -- 172.21.15.16:0/3204856980 wait complete. 2024-05-27T07:05:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:05:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:43 smithi164 ceph-mon[35327]: pgmap v147: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 37 MiB/s wr, 599 op/s 2024-05-27T07:05:44.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:43 smithi069 ceph-mon[35276]: pgmap v147: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 37 MiB/s wr, 599 op/s 2024-05-27T07:05:44.941 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3204856980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:05:44.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:05:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3204856980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:05:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3204856980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:05:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:45.989 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:45 smithi016 ceph-mon[31415]: pgmap v148: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 30 MiB/s wr, 491 op/s 2024-05-27T07:05:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:45 smithi164 ceph-mon[35327]: pgmap v148: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 30 MiB/s wr, 491 op/s 2024-05-27T07:05:46.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:45 smithi069 ceph-mon[35276]: pgmap v148: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 30 MiB/s wr, 491 op/s 2024-05-27T07:05:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:47 smithi164 ceph-mon[35327]: pgmap v149: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 44 MiB/s wr, 720 op/s 2024-05-27T07:05:48.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:47 smithi016 ceph-mon[31415]: pgmap v149: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 44 MiB/s wr, 720 op/s 2024-05-27T07:05:48.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:47 smithi069 ceph-mon[35276]: pgmap v149: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 44 MiB/s wr, 720 op/s 2024-05-27T07:05:49.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:49 smithi016 ceph-mon[31415]: pgmap v150: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:05:50.087 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:49 smithi164 ceph-mon[35327]: pgmap v150: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:05:50.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:49 smithi069 ceph-mon[35276]: pgmap v150: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:05:52.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:51 smithi164 ceph-mon[35327]: pgmap v151: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 608 op/s 2024-05-27T07:05:52.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:51 smithi016 ceph-mon[31415]: pgmap v151: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 608 op/s 2024-05-27T07:05:52.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:51 smithi069 ceph-mon[35276]: pgmap v151: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 608 op/s 2024-05-27T07:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.010+0000 7f158f200700 1 -- 172.21.15.16:0/4192134327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009380 msgr2=0x7f1588009790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.010+0000 7f158f200700 1 --2- 172.21.15.16:0/4192134327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009380 0x7f1588009790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15800098f0 tx=0x7f1580009c00 comp rx=0 tx=0).stop 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.011+0000 7f158f200700 1 -- 172.21.15.16:0/4192134327 shutdown_connections 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.011+0000 7f158f200700 1 --2- 172.21.15.16:0/4192134327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15880a7460 0x7f15880a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.011+0000 7f158f200700 1 --2- 172.21.15.16:0/4192134327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009380 0x7f1588009790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.011+0000 7f158f200700 1 --2- 172.21.15.16:0/4192134327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1588009cd0 0x7f15880a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.011+0000 7f158f200700 1 -- 172.21.15.16:0/4192134327 >> 172.21.15.16:0/4192134327 conn(0x7f15880942c0 msgr2=0x7f1588096710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.012+0000 7f158f200700 1 -- 172.21.15.16:0/4192134327 shutdown_connections 2024-05-27T07:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.012+0000 7f158f200700 1 -- 172.21.15.16:0/4192134327 wait complete. 2024-05-27T07:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.015+0000 7f158f200700 1 Processor -- start 2024-05-27T07:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.015+0000 7f158f200700 1 -- start start 2024-05-27T07:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 0x7f1588133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 0x7f1588137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15880acc00 con 0x7f15880a7460 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15880aca80 con 0x7f1588009cd0 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158f200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15880acd80 con 0x7f1588009380 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32866/0 (socket says 172.21.15.16:32866) 2024-05-27T07:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.016+0000 7f158d400700 1 -- 172.21.15.16:0/2998045284 learned_addr learned my addr 172.21.15.16:0/2998045284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.017+0000 7f158d400700 1 -- 172.21.15.16:0/2998045284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 msgr2=0x7f1588133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.017+0000 7f158de00700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 0x7f1588133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.017+0000 7f158d400700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 0x7f1588133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.017+0000 7f158d400700 1 -- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 msgr2=0x7f1588137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.018+0000 7f158e800700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 0x7f1588137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.019+0000 7f158d400700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 0x7f1588137ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.019+0000 7f158d400700 1 -- 172.21.15.16:0/2998045284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588138560 con 0x7f1588009cd0 2024-05-27T07:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.019+0000 7f158de00700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 0x7f1588133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.020+0000 7f158e800700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 0x7f1588137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.021+0000 7f158d400700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ec70 tx=0x7f157c00c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.022+0000 7f1586a00700 1 -- 172.21.15.16:0/2998045284 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c00cdc0 con 0x7f1588009cd0 2024-05-27T07:05:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.023+0000 7f1586a00700 1 -- 172.21.15.16:0/2998045284 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f157c0044b0 con 0x7f1588009cd0 2024-05-27T07:05:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.023+0000 7f1586a00700 1 -- 172.21.15.16:0/2998045284 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c010430 con 0x7f1588009cd0 2024-05-27T07:05:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.023+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1588138850 con 0x7f1588009cd0 2024-05-27T07:05:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.023+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1588151180 con 0x7f1588009cd0 2024-05-27T07:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.038+0000 7f1586a00700 1 -- 172.21.15.16:0/2998045284 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f157c0105f0 con 0x7f1588009cd0 2024-05-27T07:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.038+0000 7f15a3aefb80 1 Processor -- start 2024-05-27T07:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.039+0000 7f15a3aefb80 1 -- start start 2024-05-27T07:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aecd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b19adeee40 con 0x55b19aecb2c0 2024-05-27T07:05:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b19adde9e0 con 0x55b19ad41e40 2024-05-27T07:05:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.042+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b19ade2a20 con 0x55b19ae2d9f0 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.043+0000 7f158de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.043+0000 7f158de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32876/0 (socket says 172.21.15.16:32876) 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.043+0000 7f158de00700 1 -- 172.21.15.16:0/586858719 learned_addr learned my addr 172.21.15.16:0/586858719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.044+0000 7f1586a00700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 0x7f15740b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158de00700 1 -- 172.21.15.16:0/586858719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 msgr2=0x55b19aec1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158de00700 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158de00700 1 -- 172.21.15.16:0/586858719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 msgr2=0x55b19aecd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158de00700 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aecd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158de00700 1 -- 172.21.15.16:0/586858719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19ada7420 con 0x55b19ad41e40 2024-05-27T07:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.045+0000 7f158e800700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 0x7f15740b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.046+0000 7f1586a00700 1 -- 172.21.15.16:0/2998045284 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f157c0e2cb0 con 0x7f1588009cd0 2024-05-27T07:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.046+0000 7f158de00700 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1580009230 tx=0x7f158000fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b1ec1ba74201fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.047+0000 7f157b400700 1 -- 172.21.15.16:0/586858719 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158000fda0 con 0x55b19ad41e40 2024-05-27T07:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.047+0000 7f157b400700 1 -- 172.21.15.16:0/586858719 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f158000f8f0 con 0x55b19ad41e40 2024-05-27T07:05:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.048+0000 7f157b400700 1 -- 172.21.15.16:0/586858719 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580004930 con 0x55b19ad41e40 2024-05-27T07:05:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.049+0000 7f15a3aefb80 1 -- 172.21.15.16:0/586858719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 msgr2=0x55b19aec1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.049+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1580009230 tx=0x7f158000fb00 comp rx=0 tx=0).stop 2024-05-27T07:05:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.049+0000 7f158e800700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 0x7f15740b5050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1590051580 tx=0x7f1590064d10 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.050+0000 7f15a3aefb80 1 -- 172.21.15.16:0/586858719 shutdown_connections 2024-05-27T07:05:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.050+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aecd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.050+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.050+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/586858719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec1c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.050+0000 7f15a3aefb80 1 -- 172.21.15.16:0/586858719 >> 172.21.15.16:0/586858719 conn(0x55b19addb390 msgr2=0x55b19aebc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.053+0000 7f15a3aefb80 1 -- 172.21.15.16:0/586858719 shutdown_connections 2024-05-27T07:05:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.053+0000 7f15a3aefb80 1 -- 172.21.15.16:0/586858719 wait complete. 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.054+0000 7f15a3aefb80 1 -- start start 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.054+0000 7f15a3aefb80 1 Processor -- start 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aec8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b19ace4fa0 con 0x55b19aecb2c0 2024-05-27T07:05:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b19ad7f760 con 0x55b19ad41e40 2024-05-27T07:05:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b19aecde50 con 0x55b19ae2d9f0 2024-05-27T07:05:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.055+0000 7f15a3aefb80 10 client.0 _ll_register_callbacks cb 0x55b19ad88c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158e800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aec8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43110/0 (socket says 172.21.15.16:43110) 2024-05-27T07:05:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 -- 172.21.15.16:0/4169873849 learned_addr learned my addr 172.21.15.16:0/4169873849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 -- 172.21.15.16:0/4169873849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 msgr2=0x55b19aec81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 -- 172.21.15.16:0/4169873849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 msgr2=0x55b19aec8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aec8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.057+0000 7f158d400700 1 -- 172.21.15.16:0/4169873849 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19ad2e8d0 con 0x55b19ae2d9f0 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.058+0000 7f158d400700 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ec40 tx=0x7f157c0a9420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca3cb2e38550a2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.058+0000 7f156fe00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0a9710 con 0x55b19ae2d9f0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f157c0c4290 con 0x55b19ae2d9f0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158000f8f0 con 0x55b19ae2d9f0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f157c0ca050 con 0x55b19ae2d9f0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f15a3aefb80 10 client.25819 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.059+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b19adaf060 con 0x55b19ae2d9f0 2024-05-27T07:05:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.060+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f158000f8f0 con 0x55b19ae2d9f0 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.060+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f157c0b25d0 con 0x55b19ae2d9f0 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 10 client.25819 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 10 client.25819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 20 client.25819 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19aeebdf0 0x55b19aeee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 4 client.25819 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b19adf0d90 con 0x55b19aeebdf0 2024-05-27T07:05:53.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.061+0000 7f158de00700 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19aeebdf0 0x55b19aeee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.063+0000 7f158de00700 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19aeebdf0 0x55b19aeee210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fafd3cd6a3596ebd server_cookie=ccd5275fc8c6a2b3 in_seq=0 out_seq=0 2024-05-27T07:05:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.063+0000 7f156fe00700 10 client.25819 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.065+0000 7f156fe00700 1 -- 172.21.15.16:0/4169873849 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b19adf0d90 con 0x55b19aeebdf0 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.065+0000 7f156fe00700 10 client.25819 handle_command_reply: tid=0 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.065+0000 7f15a3aefb80 1 client.25819 shutdown 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 msgr2=0x55b19aec86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ec40 tx=0x7f157c0a9420 comp rx=0 tx=0).stop 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 shutdown_connections 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19aeebdf0 0x55b19aeee210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19aecb2c0 0x55b19aec8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19ad41e40 0x55b19aec81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/4169873849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aec86e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 >> 172.21.15.16:0/4169873849 conn(0x55b19addb390 msgr2=0x55b19aebc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f156fe00700 0 client.25819 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:05:53.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.066+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 shutdown_connections 2024-05-27T07:05:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.067+0000 7f15a3aefb80 1 -- 172.21.15.16:0/4169873849 wait complete. 2024-05-27T07:05:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.067+0000 7f15a3aefb80 20 client.25819 trim_cache size 0 max 16384 2024-05-27T07:05:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.103+0000 7f15a3aefb80 1 Processor -- start 2024-05-27T07:05:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.103+0000 7f15a3aefb80 1 -- start start 2024-05-27T07:05:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.106+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.106+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ae2d9f0 0x55b19aed6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.106+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aec3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.106+0000 7f15a3aefb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15881538d0 con 0x55b19ae2d9f0 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f158de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f158de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43112/0 (socket says 172.21.15.16:43112) 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f158de00700 1 -- 172.21.15.16:0/2499385448 learned_addr learned my addr 172.21.15.16:0/2499385448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15881516c0 con 0x55b19aecb2c0 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1588153200 con 0x55b19ad41e40 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f158d400700 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ae2d9f0 0x55b19aed6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.107+0000 7f158e800700 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aec3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.108+0000 7f158de00700 1 -- 172.21.15.16:0/2499385448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 msgr2=0x55b19aec3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.108+0000 7f158de00700 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aec3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.108+0000 7f158de00700 1 -- 172.21.15.16:0/2499385448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ae2d9f0 msgr2=0x55b19aed6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.108+0000 7f158de00700 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ae2d9f0 0x55b19aed6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.108+0000 7f158de00700 1 -- 172.21.15.16:0/2499385448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19aec8f90 con 0x55b19ad41e40 2024-05-27T07:05:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.109+0000 7f158de00700 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1580000c00 tx=0x7f15800058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d07e4ce6e603bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.109+0000 7f157aa00700 1 -- 172.21.15.16:0/2499385448 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158000f8e0 con 0x55b19ad41e40 2024-05-27T07:05:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.110+0000 7f157aa00700 1 -- 172.21.15.16:0/2499385448 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15800054d0 con 0x55b19ad41e40 2024-05-27T07:05:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.110+0000 7f157aa00700 1 -- 172.21.15.16:0/2499385448 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580004b10 con 0x55b19ad41e40 2024-05-27T07:05:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 msgr2=0x55b19aebc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1580000c00 tx=0x7f15800058e0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 shutdown_connections 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aec3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ad41e40 0x55b19aebc8b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/2499385448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ae2d9f0 0x55b19aed6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.111+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 >> 172.21.15.16:0/2499385448 conn(0x55b19addb390 msgr2=0x55b19aebc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.112+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 shutdown_connections 2024-05-27T07:05:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.112+0000 7f15a3aefb80 1 -- 172.21.15.16:0/2499385448 wait complete. 2024-05-27T07:05:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.113+0000 7f15a3aefb80 1 -- start start 2024-05-27T07:05:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.113+0000 7f15a3aefb80 1 Processor -- start 2024-05-27T07:05:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.116+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ad41e40 0x55b19aece120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.117+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aece660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.117+0000 7f15a3aefb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f158e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f158e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32910/0 (socket says 172.21.15.16:32910) 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f158e800700 1 -- 172.21.15.16:0/349038046 learned_addr learned my addr 172.21.15.16:0/349038046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1588152460 con 0x55b19ad41e40 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1588151d90 con 0x55b19aecb2c0 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1588152b30 con 0x55b19ae2d9f0 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.118+0000 7f15a3aefb80 10 client.0 _ll_register_callbacks cb 0x55b19ad88c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.119+0000 7f158e800700 1 -- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 msgr2=0x55b19aece660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.119+0000 7f158e800700 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aece660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.119+0000 7f158e800700 1 -- 172.21.15.16:0/349038046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ad41e40 msgr2=0x55b19aece120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.119+0000 7f158e800700 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ad41e40 0x55b19aece120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.119+0000 7f158e800700 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19af548d0 con 0x55b19aecb2c0 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.121+0000 7f156fe00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.121+0000 7f158e800700 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1590069440 tx=0x7f1590064ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b66f17ad97d869 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.123+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159007f6c0 con 0x55b19aecb2c0 2024-05-27T07:05:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.124+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1590079030 con 0x55b19aecb2c0 2024-05-27T07:05:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.124+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15800054d0 con 0x55b19aecb2c0 2024-05-27T07:05:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.125+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f159007f860 con 0x55b19aecb2c0 2024-05-27T07:05:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.125+0000 7f15a3aefb80 10 client.35885 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.125+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b19af55b70 con 0x55b19aecb2c0 2024-05-27T07:05:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15800054d0 con 0x55b19aecb2c0 2024-05-27T07:05:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1590098b10 con 0x55b19aecb2c0 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 10 client.35885 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 10 client.35885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 20 client.35885 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19af56250 0x55b19af58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 4 client.35885 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.127+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b19adea570 con 0x55b19af56250 2024-05-27T07:05:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.128+0000 7f158de00700 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19af56250 0x55b19af58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:53.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.130+0000 7f156fe00700 10 client.35885 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:05:53.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.131+0000 7f158de00700 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19af56250 0x55b19af58670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2334ef0c43ce8a5 server_cookie=cac3350e8097222a in_seq=0 out_seq=0 2024-05-27T07:05:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.132+0000 7f156fe00700 1 -- 172.21.15.16:0/349038046 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b19adea570 con 0x55b19af56250 2024-05-27T07:05:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.132+0000 7f156fe00700 10 client.35885 handle_command_reply: tid=0 2024-05-27T07:05:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.132+0000 7f15a3aefb80 1 client.35885 shutdown 2024-05-27T07:05:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 msgr2=0x55b19aeceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1590069440 tx=0x7f1590064ca0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 shutdown_connections 2024-05-27T07:05:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55b19af56250 0x55b19af58670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b19aecb2c0 0x55b19aeceba0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.135+0000 7f156fe00700 0 client.35885 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:05:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.136+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b19ad41e40 0x55b19aece120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.136+0000 7f15a3aefb80 1 --2- 172.21.15.16:0/349038046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b19ae2d9f0 0x55b19aece660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.136+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 >> 172.21.15.16:0/349038046 conn(0x55b19addb390 msgr2=0x55b19aebc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.136+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 shutdown_connections 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.136+0000 7f15a3aefb80 1 -- 172.21.15.16:0/349038046 wait complete. 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.137+0000 7f15a3aefb80 20 client.35885 trim_cache size 0 max 16384 2024-05-27T07:05:53.128 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:05:53.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 msgr2=0x7f15740b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 0x7f15740b5050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1590051580 tx=0x7f1590064d10 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 msgr2=0x7f1588133c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ec70 tx=0x7f157c00c5b0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 shutdown_connections 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f15740b2b40 0x7f15740b5050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15880a7460 0x7f1588137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1588009380 0x7f1588133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 --2- 172.21.15.16:0/2998045284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588009cd0 0x7f1588133c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.139+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 >> 172.21.15.16:0/2998045284 conn(0x7f15880942c0 msgr2=0x7f15880a9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.140+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 shutdown_connections 2024-05-27T07:05:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:53.140+0000 7f158f200700 1 -- 172.21.15.16:0/2998045284 wait complete. 2024-05-27T07:05:53.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:53 smithi016 ceph-mon[31415]: pgmap v152: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:05:54.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:53 smithi164 ceph-mon[35327]: pgmap v152: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:05:54.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:05:54.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:53 smithi069 ceph-mon[35276]: pgmap v152: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:05:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:05:56.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:55 smithi164 ceph-mon[35327]: pgmap v153: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 485 op/s 2024-05-27T07:05:56.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:55 smithi016 ceph-mon[31415]: pgmap v153: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 485 op/s 2024-05-27T07:05:56.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:55 smithi069 ceph-mon[35276]: pgmap v153: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 485 op/s 2024-05-27T07:05:57.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:57 smithi016 ceph-mon[31415]: pgmap v154: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 43 MiB/s wr, 705 op/s 2024-05-27T07:05:58.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:57 smithi164 ceph-mon[35327]: pgmap v154: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 43 MiB/s wr, 705 op/s 2024-05-27T07:05:58.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:57 smithi069 ceph-mon[35276]: pgmap v154: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 43 MiB/s wr, 705 op/s 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.042+0000 7f0471800700 1 -- 172.21.15.16:0/3384957030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 msgr2=0x7f046c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.042+0000 7f0471800700 1 --2- 172.21.15.16:0/3384957030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c1111c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0464009990 tx=0x7f0464009ca0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.043+0000 7f0471800700 1 -- 172.21.15.16:0/3384957030 shutdown_connections 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.043+0000 7f0471800700 1 --2- 172.21.15.16:0/3384957030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c1111c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.043+0000 7f0471800700 1 --2- 172.21.15.16:0/3384957030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c1045e0 0x7f046c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.043+0000 7f0471800700 1 --2- 172.21.15.16:0/3384957030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c108f10 0x7f046c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.043+0000 7f0471800700 1 -- 172.21.15.16:0/3384957030 >> 172.21.15.16:0/3384957030 conn(0x7f046c100030 msgr2=0x7f046c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.044+0000 7f0471800700 1 -- 172.21.15.16:0/3384957030 shutdown_connections 2024-05-27T07:05:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.044+0000 7f0471800700 1 -- 172.21.15.16:0/3384957030 wait complete. 2024-05-27T07:05:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.046+0000 7f0471800700 1 Processor -- start 2024-05-27T07:05:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.046+0000 7f0471800700 1 -- start start 2024-05-27T07:05:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.047+0000 7f0471800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.047+0000 7f0471800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 0x7f046c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.047+0000 7f0471800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f0471800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f046c114410 con 0x7f046c108f10 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f0471800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f046c114290 con 0x7f046c109860 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f0471800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f046c114590 con 0x7f046c1045e0 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f046b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f046aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f046aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43118/0 (socket says 172.21.15.16:43118) 2024-05-27T07:05:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f046b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32936/0 (socket says 172.21.15.16:32936) 2024-05-27T07:05:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.048+0000 7f046aa00700 1 -- 172.21.15.16:0/5748370 learned_addr learned my addr 172.21.15.16:0/5748370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.049+0000 7f0462000700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 0x7f046c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f046aa00700 1 -- 172.21.15.16:0/5748370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 msgr2=0x7f046c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f046aa00700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f046aa00700 1 -- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 msgr2=0x7f046c19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f046aa00700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 0x7f046c19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f046aa00700 1 -- 172.21.15.16:0/5748370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c19fdf0 con 0x7f046c1045e0 2024-05-27T07:05:59.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.050+0000 7f0462000700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 0x7f046c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:05:59.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.052+0000 7f046aa00700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f045400b530 tx=0x7f045400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.053+0000 7f0463e00700 1 -- 172.21.15.16:0/5748370 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454015020 con 0x7f046c1045e0 2024-05-27T07:05:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.054+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1a00e0 con 0x7f046c1045e0 2024-05-27T07:05:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.054+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a7a00 con 0x7f046c1045e0 2024-05-27T07:05:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.054+0000 7f0463e00700 1 -- 172.21.15.16:0/5748370 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0454011cc0 con 0x7f046c1045e0 2024-05-27T07:05:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.054+0000 7f0463e00700 1 -- 172.21.15.16:0/5748370 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454010d30 con 0x7f046c1045e0 2024-05-27T07:05:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.061+0000 7f0463e00700 1 -- 172.21.15.16:0/5748370 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0454018020 con 0x7f046c1045e0 2024-05-27T07:05:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.065+0000 7f0463e00700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 0x7f04500b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.065+0000 7f0463e00700 1 -- 172.21.15.16:0/5748370 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f04540e1f80 con 0x7f046c1045e0 2024-05-27T07:05:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.066+0000 7f0462000700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 0x7f04500b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.066+0000 7f0480413b80 1 Processor -- start 2024-05-27T07:05:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 -- start start 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x564891393910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5648912c1e40 con 0x5648913009f0 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648912b19e0 con 0x564891214e40 2024-05-27T07:05:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.067+0000 7f0480413b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5648912b5a20 con 0x5648913a2510 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.068+0000 7f046b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.068+0000 7f046b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43132/0 (socket says 172.21.15.16:43132) 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.068+0000 7f046b400700 1 -- 172.21.15.16:0/2160767188 learned_addr learned my addr 172.21.15.16:0/2160767188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.069+0000 7f046b400700 1 -- 172.21.15.16:0/2160767188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 msgr2=0x564891393910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.069+0000 7f046b400700 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x564891393910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.069+0000 7f046b400700 1 -- 172.21.15.16:0/2160767188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 msgr2=0x5648913933d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:05:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.069+0000 7f046b400700 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.069+0000 7f046b400700 1 -- 172.21.15.16:0/2160767188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56489127a420 con 0x5648913a2510 2024-05-27T07:05:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.070+0000 7f0462000700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 0x7f04500b50d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f046c1035d0 tx=0x7f045c00b040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.072+0000 7f046b400700 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f046400b6f0 tx=0x7f0464005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e09e2a8fddbd6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.074+0000 7f045be00700 1 -- 172.21.15.16:0/2160767188 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640238d0 con 0x5648913a2510 2024-05-27T07:05:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.074+0000 7f045be00700 1 -- 172.21.15.16:0/2160767188 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0464027030 con 0x5648913a2510 2024-05-27T07:05:59.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.074+0000 7f045be00700 1 -- 172.21.15.16:0/2160767188 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464003680 con 0x5648913a2510 2024-05-27T07:05:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 -- 172.21.15.16:0/2160767188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 msgr2=0x5648913a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f046400b6f0 tx=0x7f0464005040 comp rx=0 tx=0).stop 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 -- 172.21.15.16:0/2160767188 shutdown_connections 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x5648913a4930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913933d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 --2- 172.21.15.16:0/2160767188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x564891393910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.075+0000 7f0480413b80 1 -- 172.21.15.16:0/2160767188 >> 172.21.15.16:0/2160767188 conn(0x5648912ae390 msgr2=0x56489138f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.076+0000 7f0480413b80 1 -- 172.21.15.16:0/2160767188 shutdown_connections 2024-05-27T07:05:59.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.076+0000 7f0480413b80 1 -- 172.21.15.16:0/2160767188 wait complete. 2024-05-27T07:05:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.077+0000 7f0480413b80 1 -- start start 2024-05-27T07:05:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.078+0000 7f0480413b80 1 Processor -- start 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5648911b7fa0 con 0x5648913009f0 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564891252760 con 0x564891214e40 2024-05-27T07:05:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.079+0000 7f0480413b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564891396190 con 0x5648913a2510 2024-05-27T07:05:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046aa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43136/0 (socket says 172.21.15.16:43136) 2024-05-27T07:05:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 -- 172.21.15.16:0/2025326414 learned_addr learned my addr 172.21.15.16:0/2025326414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f0480413b80 10 client.0 _ll_register_callbacks cb 0x56489125bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 -- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 msgr2=0x5648913a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 -- 172.21.15.16:0/2025326414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 msgr2=0x5648913a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.080+0000 7f046b400700 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.081+0000 7f046b400700 1 -- 172.21.15.16:0/2025326414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648912018d0 con 0x5648913a2510 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.081+0000 7f046b400700 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0464005b40 tx=0x7f046400b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413409e4fa1c1307 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.082+0000 7f0458c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.082+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464023ad0 con 0x5648913a2510 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.082+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f0464023c70 con 0x5648913a2510 2024-05-27T07:05:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.084+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0464027030 con 0x5648913a2510 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.084+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f046403b050 con 0x5648913a2510 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.084+0000 7f0480413b80 10 client.25834 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.084+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564891282060 con 0x5648913a2510 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.085+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0464027030 con 0x5648913a2510 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.085+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f046400f660 con 0x5648913a2510 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.085+0000 7f0480413b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.086+0000 7f0480413b80 10 client.25834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:05:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.086+0000 7f0480413b80 20 client.25834 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.086+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5648913adb30 0x5648913aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.086+0000 7f0480413b80 4 client.25834 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:05:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.086+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648912c3d90 con 0x5648913adb30 2024-05-27T07:05:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.090+0000 7f046aa00700 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5648913adb30 0x5648913aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.092+0000 7f046aa00700 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5648913adb30 0x5648913aff50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea67c358daf8c54e server_cookie=110cb6e8ef466d7e in_seq=0 out_seq=0 2024-05-27T07:05:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.092+0000 7f0458c00700 10 client.25834 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:05:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.095+0000 7f0458c00700 1 -- 172.21.15.16:0/2025326414 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5648912c3d90 con 0x5648913adb30 2024-05-27T07:05:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.095+0000 7f0458c00700 10 client.25834 handle_command_reply: tid=0 2024-05-27T07:05:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.095+0000 7f0480413b80 1 client.25834 shutdown 2024-05-27T07:05:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 msgr2=0x56489139dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0464005b40 tx=0x7f046400b750 comp rx=0 tx=0).stop 2024-05-27T07:05:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 shutdown_connections 2024-05-27T07:05:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913a2510 0x56489139dbc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5648913adb30 0x5648913aff50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 --2- 172.21.15.16:0/2025326414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913009f0 0x5648913a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0458c00700 0 client.25834 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:05:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.097+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 >> 172.21.15.16:0/2025326414 conn(0x5648912ae390 msgr2=0x564891393410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.099+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 shutdown_connections 2024-05-27T07:05:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.099+0000 7f0480413b80 1 -- 172.21.15.16:0/2025326414 wait complete. 2024-05-27T07:05:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.100+0000 7f0480413b80 20 client.25834 trim_cache size 0 max 16384 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 Processor -- start 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 -- start start 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x56489138f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913a2510 0x56489139d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f044800edf0 con 0x5648913a2510 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0448010930 con 0x564891214e40 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.134+0000 7f0480413b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f044800f4c0 con 0x5648913009f0 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.135+0000 7f046aa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.135+0000 7f046aa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:32956/0 (socket says 172.21.15.16:32956) 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.135+0000 7f046aa00700 1 -- 172.21.15.16:0/2243106885 learned_addr learned my addr 172.21.15.16:0/2243106885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.135+0000 7f0462000700 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x56489138f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 -- 172.21.15.16:0/2243106885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 msgr2=0x56489138f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x56489138f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 -- 172.21.15.16:0/2243106885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913a2510 msgr2=0x56489139d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913a2510 0x56489139d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 -- 172.21.15.16:0/2243106885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56489139e9a0 con 0x564891214e40 2024-05-27T07:05:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.136+0000 7f046aa00700 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04540a8f10 tx=0x7f045400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf95b0e65b02ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.137+0000 7f045b400700 1 -- 172.21.15.16:0/2243106885 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04540b8e60 con 0x564891214e40 2024-05-27T07:05:59.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.138+0000 7f045b400700 1 -- 172.21.15.16:0/2243106885 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04540a8f80 con 0x564891214e40 2024-05-27T07:05:59.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.138+0000 7f045b400700 1 -- 172.21.15.16:0/2243106885 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04540c3030 con 0x564891214e40 2024-05-27T07:05:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.139+0000 7f0480413b80 1 -- 172.21.15.16:0/2243106885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 msgr2=0x5648913993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.140+0000 7f0480413b80 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04540a8f10 tx=0x7f045400bf20 comp rx=0 tx=0).stop 2024-05-27T07:05:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 -- 172.21.15.16:0/2243106885 shutdown_connections 2024-05-27T07:05:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648913a2510 0x56489139d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564891214e40 0x5648913993c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 --2- 172.21.15.16:0/2243106885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x56489138f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 -- 172.21.15.16:0/2243106885 >> 172.21.15.16:0/2243106885 conn(0x5648912ae390 msgr2=0x56489138ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 -- 172.21.15.16:0/2243106885 shutdown_connections 2024-05-27T07:05:59.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.141+0000 7f0480413b80 1 -- 172.21.15.16:0/2243106885 wait complete. 2024-05-27T07:05:59.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.142+0000 7f0480413b80 1 -- start start 2024-05-27T07:05:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.143+0000 7f0480413b80 1 Processor -- start 2024-05-27T07:05:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.145+0000 7f0480413b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 0x5648913a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.145+0000 7f0480413b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x5648913a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.146+0000 7f0480413b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 0x5648913a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.146+0000 7f046aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 0x5648913a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.146+0000 7f0462000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x5648913a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.146+0000 7f0462000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x5648913a52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43154/0 (socket says 172.21.15.16:43154) 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.146+0000 7f046aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 0x5648913a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43712/0 (socket says 172.21.15.16:43712) 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f0462000700 1 -- 172.21.15.16:0/1006390712 learned_addr learned my addr 172.21.15.16:0/1006390712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0448010260 con 0x564891214e40 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f044800fb90 con 0x5648913a2510 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f044800cd90 con 0x5648913009f0 2024-05-27T07:05:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f0480413b80 10 client.0 _ll_register_callbacks cb 0x56489125bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:05:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.147+0000 7f046b400700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 0x5648913a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f046b400700 1 -- 172.21.15.16:0/1006390712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 msgr2=0x5648913a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f046b400700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x5648913a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f046b400700 1 -- 172.21.15.16:0/1006390712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 msgr2=0x5648913a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:05:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f046b400700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 0x5648913a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f046b400700 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648914274c0 con 0x5648913a2510 2024-05-27T07:05:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.148+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04540a8f80 con 0x5648913a2510 2024-05-27T07:05:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.149+0000 7f046b400700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 0x5648913a5820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0464005f50 tx=0x7f046400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c3e51f76e060fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:05:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.149+0000 7f0458c00700 10 client.35897 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:05:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.151+0000 7f0458c00700 1 -- 172.21.15.16:0/1006390712 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464023ae0 con 0x5648913a2510 2024-05-27T07:05:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.151+0000 7f0458c00700 1 -- 172.21.15.16:0/1006390712 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f0464023c80 con 0x5648913a2510 2024-05-27T07:05:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.151+0000 7f0458c00700 1 -- 172.21.15.16:0/1006390712 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0464042930 con 0x5648913a2510 2024-05-27T07:05:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.151+0000 7f0480413b80 10 client.35897 fetch_fsmap learned FSMap version 31 2024-05-27T07:05:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.151+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x564891428620 con 0x5648913a2510 2024-05-27T07:05:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.153+0000 7f0458c00700 1 -- 172.21.15.16:0/1006390712 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0464042ad0 con 0x5648913a2510 2024-05-27T07:05:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 10 client.35897 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:05:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 10 client.35897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:05:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 20 client.35897 populate_metadata read hostname 'smithi016' 2024-05-27T07:05:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x564891428b60 0x56489142afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:05:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 4 client.35897 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:05:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648912bd570 con 0x564891428b60 2024-05-27T07:05:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.154+0000 7f046aa00700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x564891428b60 0x56489142afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:05:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.156+0000 7f046aa00700 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x564891428b60 0x56489142afa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6f50a0446f7fba9 server_cookie=dc786e495b42f062 in_seq=0 out_seq=0 2024-05-27T07:05:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.156+0000 7f0458c00700 10 client.35897 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:05:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.156+0000 7f0458c00700 1 -- 172.21.15.16:0/1006390712 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648912bd570 con 0x564891428b60 2024-05-27T07:05:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0458c00700 10 client.35897 handle_command_reply: tid=0 2024-05-27T07:05:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 client.35897 shutdown 2024-05-27T07:05:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 msgr2=0x5648913a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 0x5648913a5820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0464005f50 tx=0x7f046400f7f0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 shutdown_connections 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x564891428b60 0x56489142afa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648913a2510 0x5648913a5820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564891214e40 0x5648913a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 --2- 172.21.15.16:0/1006390712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648913009f0 0x5648913a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0458c00700 0 client.35897 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.157+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 >> 172.21.15.16:0/1006390712 conn(0x5648912ae390 msgr2=0x56489139d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.159+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 shutdown_connections 2024-05-27T07:05:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.159+0000 7f0480413b80 1 -- 172.21.15.16:0/1006390712 wait complete. 2024-05-27T07:05:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.159+0000 7f0480413b80 20 client.35897 trim_cache size 0 max 16384 2024-05-27T07:05:59.150 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:05:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 msgr2=0x7f04500b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 0x7f04500b50d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f046c1035d0 tx=0x7f045c00b040 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 msgr2=0x7f046c19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f045400b530 tx=0x7f045400b8f0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 shutdown_connections 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04500b2bc0 0x7f04500b50d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f046c109860 0x7f046c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f046c1045e0 0x7f046c19af20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 --2- 172.21.15.16:0/5748370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c108f10 0x7f046c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.160+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 >> 172.21.15.16:0/5748370 conn(0x7f046c100030 msgr2=0x7f046c108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.161+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 shutdown_connections 2024-05-27T07:05:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:05:59.161+0000 7f0471800700 1 -- 172.21.15.16:0/5748370 wait complete. 2024-05-27T07:06:00.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:05:59 smithi164 ceph-mon[35327]: pgmap v155: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 475 op/s 2024-05-27T07:06:00.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:05:59 smithi016 ceph-mon[31415]: pgmap v155: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 475 op/s 2024-05-27T07:06:00.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:05:59 smithi069 ceph-mon[35276]: pgmap v155: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 475 op/s 2024-05-27T07:06:00.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:06:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:01.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:01 smithi016 ceph-mon[31415]: pgmap v156: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 37 MiB/s wr, 605 op/s 2024-05-27T07:06:02.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:01 smithi164 ceph-mon[35327]: pgmap v156: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 37 MiB/s wr, 605 op/s 2024-05-27T07:06:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:01 smithi069 ceph-mon[35276]: pgmap v156: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 37 MiB/s wr, 605 op/s 2024-05-27T07:06:03.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:03 smithi016 ceph-mon[31415]: pgmap v157: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:06:04.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:03 smithi164 ceph-mon[35327]: pgmap v157: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:06:04.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:03 smithi069 ceph-mon[35276]: pgmap v157: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 607 op/s 2024-05-27T07:06:06.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:05 smithi164 ceph-mon[35327]: pgmap v158: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:06:06.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:05 smithi016 ceph-mon[31415]: pgmap v158: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:06:06.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:05 smithi069 ceph-mon[35276]: pgmap v158: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 479 op/s 2024-05-27T07:06:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:07 smithi164 ceph-mon[35327]: pgmap v159: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 688 op/s 2024-05-27T07:06:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:07 smithi016 ceph-mon[31415]: pgmap v159: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 688 op/s 2024-05-27T07:06:08.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:07 smithi069 ceph-mon[35276]: pgmap v159: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 688 op/s 2024-05-27T07:06:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.200+0000 7f6c5fe00700 1 -- 172.21.15.16:0/634528891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6006fd40 msgr2=0x7f6c6011d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.200+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/634528891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6006fd40 0x7f6c6011d2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009990 tx=0x7f6c50009ca0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 -- 172.21.15.16:0/634528891 shutdown_connections 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/634528891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6011d820 0x7f6c6011fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/634528891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 0x7f6c6006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/634528891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6006fd40 0x7f6c6011d2e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 -- 172.21.15.16:0/634528891 >> 172.21.15.16:0/634528891 conn(0x7f6c6010a980 msgr2=0x7f6c6010cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 -- 172.21.15.16:0/634528891 shutdown_connections 2024-05-27T07:06:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.201+0000 7f6c5fe00700 1 -- 172.21.15.16:0/634528891 wait complete. 2024-05-27T07:06:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.205+0000 7f6c5fe00700 1 Processor -- start 2024-05-27T07:06:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.205+0000 7f6c5fe00700 1 -- start start 2024-05-27T07:06:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.206+0000 7f6c5fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 0x7f6c601a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.206+0000 7f6c5fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 0x7f6c601a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.206+0000 7f6c5fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 0x7f6c601adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.207+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 0x7f6c601a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.207+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 0x7f6c601a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:45856/0 (socket says 172.21.15.16:45856) 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.207+0000 7f6c5e000700 1 -- 172.21.15.16:0/1749025737 learned_addr learned my addr 172.21.15.16:0/1749025737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.207+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 0x7f6c601a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.207+0000 7f6c5f400700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 0x7f6c601adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c60122f60 con 0x7f6c6006f3f0 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c60122de0 con 0x7f6c6006fd40 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601230e0 con 0x7f6c6011d820 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5f400700 1 -- 172.21.15.16:0/1749025737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 msgr2=0x7f6c601a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5f400700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 0x7f6c601a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5f400700 1 -- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 msgr2=0x7f6c601a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5f400700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 0x7f6c601a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.208+0000 7f6c5f400700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c601ae660 con 0x7f6c6011d820 2024-05-27T07:06:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.209+0000 7f6c5f400700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 0x7f6c601adfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800efb0 tx=0x7f6c5800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.210+0000 7f6c57400700 1 -- 172.21.15.16:0/1749025737 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800ca50 con 0x7f6c6011d820 2024-05-27T07:06:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.210+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c601ae950 con 0x7f6c6011d820 2024-05-27T07:06:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.210+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c601b6270 con 0x7f6c6011d820 2024-05-27T07:06:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.211+0000 7f6c57400700 1 -- 172.21.15.16:0/1749025737 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c5800cbf0 con 0x7f6c6011d820 2024-05-27T07:06:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.211+0000 7f6c57400700 1 -- 172.21.15.16:0/1749025737 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800ca50 con 0x7f6c6011d820 2024-05-27T07:06:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.218+0000 7f6c57400700 1 -- 172.21.15.16:0/1749025737 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6c58012050 con 0x7f6c6011d820 2024-05-27T07:06:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.220+0000 7f6c57400700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 0x7f6c440b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.221+0000 7f6c57400700 1 -- 172.21.15.16:0/1749025737 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6c580e1e70 con 0x7f6c6011d820 2024-05-27T07:06:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.222+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 0x7f6c440b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.225+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 0x7f6c440b5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005950 tx=0x7f6c480058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.227+0000 7f6c74323b80 1 Processor -- start 2024-05-27T07:06:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.227+0000 7f6c74323b80 1 -- start start 2024-05-27T07:06:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.228+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.228+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7820 0x55cc2bce4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c74323b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bc05e40 con 0x55cc2bb58e40 2024-05-27T07:06:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c74323b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bbf59e0 con 0x55cc2bcd7820 2024-05-27T07:06:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c74323b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bbf9a20 con 0x55cc2bcd7410 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47348/0 (socket says 172.21.15.16:47348) 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c5ea00700 1 -- 172.21.15.16:0/4177767355 learned_addr learned my addr 172.21.15.16:0/4177767355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.229+0000 7f6c5e000700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5f400700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7820 0x55cc2bce4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5e000700 1 -- 172.21.15.16:0/4177767355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7820 msgr2=0x55cc2bce4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5e000700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7820 0x55cc2bce4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5e000700 1 -- 172.21.15.16:0/4177767355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 msgr2=0x55cc2bcd6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5e000700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5e000700 1 -- 172.21.15.16:0/4177767355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2bbbe420 con 0x55cc2bcd7410 2024-05-27T07:06:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.230+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.231+0000 7f6c5e000700 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c500098e0 tx=0x7f6c50005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6556eff7c7eeb43c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.232+0000 7f6c4fe00700 1 -- 172.21.15.16:0/4177767355 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50027070 con 0x55cc2bcd7410 2024-05-27T07:06:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.233+0000 7f6c4fe00700 1 -- 172.21.15.16:0/4177767355 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c50003d10 con 0x55cc2bcd7410 2024-05-27T07:06:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.233+0000 7f6c4fe00700 1 -- 172.21.15.16:0/4177767355 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000be40 con 0x55cc2bcd7410 2024-05-27T07:06:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.235+0000 7f6c74323b80 1 -- 172.21.15.16:0/4177767355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 msgr2=0x55cc2bce1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.235+0000 7f6c74323b80 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c500098e0 tx=0x7f6c50005050 comp rx=0 tx=0).stop 2024-05-27T07:06:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.235+0000 7f6c74323b80 1 -- 172.21.15.16:0/4177767355 shutdown_connections 2024-05-27T07:06:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7820 0x55cc2bce4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bcd6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 --2- 172.21.15.16:0/4177767355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 -- 172.21.15.16:0/4177767355 >> 172.21.15.16:0/4177767355 conn(0x55cc2bbb8fc0 msgr2=0x55cc2bcd2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 -- 172.21.15.16:0/4177767355 shutdown_connections 2024-05-27T07:06:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.237+0000 7f6c74323b80 1 -- 172.21.15.16:0/4177767355 wait complete. 2024-05-27T07:06:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.238+0000 7f6c74323b80 1 -- start start 2024-05-27T07:06:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.239+0000 7f6c74323b80 1 Processor -- start 2024-05-27T07:06:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.240+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bce1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c74323b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bafbfa0 con 0x55cc2bb58e40 2024-05-27T07:06:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c74323b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bb96760 con 0x55cc2bcd7410 2024-05-27T07:06:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c74323b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cc2bce4930 con 0x55cc2bcd7820 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c5f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c5f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42522/0 (socket says 172.21.15.16:42522) 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.241+0000 7f6c5f400700 1 -- 172.21.15.16:0/2188303481 learned_addr learned my addr 172.21.15.16:0/2188303481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c5f400700 1 -- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 msgr2=0x55cc2bce1a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c5f400700 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c5f400700 1 -- 172.21.15.16:0/2188303481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 msgr2=0x55cc2bce1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c5f400700 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bce1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c5f400700 1 -- 172.21.15.16:0/2188303481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2bb458d0 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.243+0000 7f6c5f400700 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009da0 tx=0x7f6c580a8970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c85c6feb3826ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.243+0000 7f6c4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.242+0000 7f6c74323b80 10 client.0 _ll_register_callbacks cb 0x55cc2bb9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.243+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c50003d10 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.243+0000 7f6c4cc00700 1 -- 172.21.15.16:0/2188303481 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580a8df0 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.243+0000 7f6c4cc00700 1 -- 172.21.15.16:0/2188303481 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6c580b1940 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c4cc00700 1 -- 172.21.15.16:0/2188303481 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c580ad1a0 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 10 client.25852 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55cc2bbc6060 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c4cc00700 1 -- 172.21.15.16:0/2188303481 <== mon.1 v2:172.21.15.164:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6c580c1030 con 0x55cc2bcd7820 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 10 client.25852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 20 client.25852 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bcf1720 0x55cc2bcf3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 4 client.25852 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.244+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc2bc07d90 con 0x55cc2bcf1720 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.245+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bcf1720 0x55cc2bcf3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.246+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bcf1720 0x55cc2bcf3ba0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bdf80d86d8c7383 server_cookie=58ee224610611c0d in_seq=0 out_seq=0 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.246+0000 7f6c4cc00700 10 client.25852 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:06:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.247+0000 7f6c4cc00700 1 -- 172.21.15.16:0/2188303481 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cc2bc07d90 con 0x55cc2bcf1720 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.247+0000 7f6c4cc00700 10 client.25852 handle_command_reply: tid=0 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 client.25852 shutdown 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 msgr2=0x55cc2bce1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009da0 tx=0x7f6c580a8970 comp rx=0 tx=0).stop 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 shutdown_connections 2024-05-27T07:06:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bcf1720 0x55cc2bcf3ba0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce1fa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bb58e40 0x55cc2bce1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2188303481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c4cc00700 0 client.25852 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.248+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 >> 172.21.15.16:0/2188303481 conn(0x55cc2bbb8fc0 msgr2=0x55cc2bcd1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.249+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 shutdown_connections 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.250+0000 7f6c74323b80 1 -- 172.21.15.16:0/2188303481 wait complete. 2024-05-27T07:06:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.250+0000 7f6c74323b80 20 client.25852 trim_cache size 0 max 16384 2024-05-27T07:06:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.278+0000 7f6c74323b80 1 Processor -- start 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.278+0000 7f6c74323b80 1 -- start start 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 0x55cc2bceda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c00ed70 con 0x55cc2bcd7820 2024-05-27T07:06:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0108b0 con 0x55cc2bcd7410 2024-05-27T07:06:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.279+0000 7f6c74323b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c00f440 con 0x55cc2bb58e40 2024-05-27T07:06:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42532/0 (socket says 172.21.15.16:42532) 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5ea00700 1 -- 172.21.15.16:0/2813863137 learned_addr learned my addr 172.21.15.16:0/2813863137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5f400700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 0x55cc2bceda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.280+0000 7f6c5ea00700 1 -- 172.21.15.16:0/2813863137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 msgr2=0x55cc2bce39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5ea00700 1 -- 172.21.15.16:0/2813863137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 msgr2=0x55cc2bceda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 0x55cc2bceda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5ea00700 1 -- 172.21.15.16:0/2813863137 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2bccd1d0 con 0x55cc2bb58e40 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5e000700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.281+0000 7f6c5f400700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 0x55cc2bceda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.282+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cc2bce0da0 tx=0x7f6c48012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a140f21cdafce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.282+0000 7f6c4f400700 1 -- 172.21.15.16:0/2813863137 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48020d80 con 0x55cc2bb58e40 2024-05-27T07:06:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.282+0000 7f6c4f400700 1 -- 172.21.15.16:0/2813863137 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c48028840 con 0x55cc2bb58e40 2024-05-27T07:06:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.282+0000 7f6c4f400700 1 -- 172.21.15.16:0/2813863137 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480117c0 con 0x55cc2bb58e40 2024-05-27T07:06:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 -- 172.21.15.16:0/2813863137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 msgr2=0x55cc2bcdcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cc2bce0da0 tx=0x7f6c48012000 comp rx=0 tx=0).stop 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 -- 172.21.15.16:0/2813863137 shutdown_connections 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7820 0x55cc2bceda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bb58e40 0x55cc2bcdcd90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 --2- 172.21.15.16:0/2813863137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bcd7410 0x55cc2bce39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.284+0000 7f6c74323b80 1 -- 172.21.15.16:0/2813863137 >> 172.21.15.16:0/2813863137 conn(0x55cc2bbb8fc0 msgr2=0x55cc2bce1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.285+0000 7f6c74323b80 1 -- 172.21.15.16:0/2813863137 shutdown_connections 2024-05-27T07:06:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.285+0000 7f6c74323b80 1 -- 172.21.15.16:0/2813863137 wait complete. 2024-05-27T07:06:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.286+0000 7f6c74323b80 1 -- start start 2024-05-27T07:06:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.286+0000 7f6c74323b80 1 Processor -- start 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.287+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 0x55cc2bd54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7410 0x55cc2bd54f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0101e0 con 0x55cc2bcd7410 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c00fb10 con 0x55cc2bb58e40 2024-05-27T07:06:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c00cd10 con 0x55cc2bcd7820 2024-05-27T07:06:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c74323b80 10 client.0 _ll_register_callbacks cb 0x55cc2bb9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.288+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 0x55cc2bd54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.289+0000 7f6c5ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 0x55cc2bd54a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:45900/0 (socket says 172.21.15.16:45900) 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.289+0000 7f6c5ea00700 1 -- 172.21.15.16:0/431148899 learned_addr learned my addr 172.21.15.16:0/431148899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.289+0000 7f6c5f400700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.289+0000 7f6c5e000700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7410 0x55cc2bd54f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c5f400700 1 -- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 msgr2=0x55cc2bd54a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c5f400700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 0x55cc2bd54a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c5f400700 1 -- 172.21.15.16:0/431148899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7410 msgr2=0x55cc2bd54f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c5f400700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7410 0x55cc2bd54f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c5f400700 1 -- 172.21.15.16:0/431148899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2bce91d0 con 0x55cc2bcd7820 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.290+0000 7f6c4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:06:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.291+0000 7f6c5f400700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce8c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c580a8af0 tx=0x7f6c580094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3f4c9f8c82785e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.292+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580a8050 con 0x55cc2bcd7820 2024-05-27T07:06:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.293+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c48028840 con 0x55cc2bcd7820 2024-05-27T07:06:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.293+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6c58012de0 con 0x55cc2bcd7820 2024-05-27T07:06:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.293+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c580a8050 con 0x55cc2bcd7820 2024-05-27T07:06:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.293+0000 7f6c74323b80 10 client.25858 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.293+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc2bcea3d0 con 0x55cc2bcd7820 2024-05-27T07:06:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.294+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c58088030 con 0x55cc2bcd7820 2024-05-27T07:06:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.294+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6c580c1040 con 0x55cc2bcd7820 2024-05-27T07:06:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.294+0000 7f6c74323b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.294+0000 7f6c74323b80 10 client.25858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:06:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.294+0000 7f6c74323b80 20 client.25858 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.295+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bd6d280 0x55cc2bd6f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.295+0000 7f6c74323b80 4 client.25858 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:06:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.295+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc2bc01570 con 0x55cc2bd6d280 2024-05-27T07:06:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.295+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bd6d280 0x55cc2bd6f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.298+0000 7f6c5ea00700 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bd6d280 0x55cc2bd6f6a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6dd6bff2a47bf1f server_cookie=a58da5455686cbf3 in_seq=0 out_seq=0 2024-05-27T07:06:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.298+0000 7f6c4cc00700 10 client.25858 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:06:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.299+0000 7f6c4cc00700 1 -- 172.21.15.16:0/431148899 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc2bc01570 con 0x55cc2bd6d280 2024-05-27T07:06:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.299+0000 7f6c4cc00700 10 client.25858 handle_command_reply: tid=0 2024-05-27T07:06:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.299+0000 7f6c74323b80 1 client.25858 shutdown 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 msgr2=0x55cc2bce8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce8c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c580a8af0 tx=0x7f6c580094e0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 shutdown_connections 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55cc2bd6d280 0x55cc2bd6f6a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc2bcd7820 0x55cc2bce8c10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c4cc00700 0 client.25858 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc2bb58e40 0x55cc2bd54a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 --2- 172.21.15.16:0/431148899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2bcd7410 0x55cc2bd54f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 >> 172.21.15.16:0/431148899 conn(0x55cc2bbb8fc0 msgr2=0x55cc2bce1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.301+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 shutdown_connections 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.302+0000 7f6c74323b80 1 -- 172.21.15.16:0/431148899 wait complete. 2024-05-27T07:06:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.302+0000 7f6c74323b80 20 client.25858 trim_cache size 0 max 16384 2024-05-27T07:06:09.295 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:06:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 msgr2=0x7f6c440b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 0x7f6c440b5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005950 tx=0x7f6c480058e0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 msgr2=0x7f6c601adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 0x7f6c601adfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800efb0 tx=0x7f6c5800c3a0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 shutdown_connections 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6c440b2cf0 0x7f6c440b5200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6011d820 0x7f6c601adfc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6006f3f0 0x7f6c601a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 --2- 172.21.15.16:0/1749025737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c6006fd40 0x7f6c601a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 >> 172.21.15.16:0/1749025737 conn(0x7f6c6010a980 msgr2=0x7f6c6011df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.303+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 shutdown_connections 2024-05-27T07:06:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:09.304+0000 7f6c5fe00700 1 -- 172.21.15.16:0/1749025737 wait complete. 2024-05-27T07:06:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:09 smithi164 ceph-mon[35327]: pgmap v160: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 499 op/s 2024-05-27T07:06:10.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:09 smithi016 ceph-mon[31415]: pgmap v160: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 499 op/s 2024-05-27T07:06:10.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:09 smithi069 ceph-mon[35276]: pgmap v160: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 499 op/s 2024-05-27T07:06:10.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:06:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:12.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:11 smithi164 ceph-mon[35327]: pgmap v161: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 583 op/s 2024-05-27T07:06:12.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:11 smithi016 ceph-mon[31415]: pgmap v161: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 583 op/s 2024-05-27T07:06:12.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:11 smithi069 ceph-mon[35276]: pgmap v161: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 583 op/s 2024-05-27T07:06:13.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:12 smithi164 ceph-mon[35327]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 577 op/s 2024-05-27T07:06:13.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:12 smithi016 ceph-mon[31415]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 577 op/s 2024-05-27T07:06:13.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:12 smithi069 ceph-mon[35276]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 577 op/s 2024-05-27T07:06:15.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:14 smithi016 ceph-mon[31415]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 447 op/s 2024-05-27T07:06:15.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:14 smithi164 ceph-mon[35327]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 447 op/s 2024-05-27T07:06:15.372 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:14 smithi069 ceph-mon[35276]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 447 op/s 2024-05-27T07:06:17.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:16 smithi016 ceph-mon[31415]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:17.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:16 smithi164 ceph-mon[35327]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:17.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:16 smithi069 ceph-mon[35276]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:17.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 -- 172.21.15.16:0/1641520194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a28110290 msgr2=0x7f8a28117bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 --2- 172.21.15.16:0/1641520194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a28110290 0x7f8a28117bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009fc0 tx=0x7f8a2000b330 comp rx=0 tx=0).stop 2024-05-27T07:06:17.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 -- 172.21.15.16:0/1641520194 shutdown_connections 2024-05-27T07:06:17.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 --2- 172.21.15.16:0/1641520194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a28110290 0x7f8a28117bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 --2- 172.21.15.16:0/1641520194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810b010 0x7f8a2810b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 --2- 172.21.15.16:0/1641520194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a2810f940 0x7f8a2810fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.754+0000 7f8a30600700 1 -- 172.21.15.16:0/1641520194 >> 172.21.15.16:0/1641520194 conn(0x7f8a2806f980 msgr2=0x7f8a2806fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.755+0000 7f8a30600700 1 -- 172.21.15.16:0/1641520194 shutdown_connections 2024-05-27T07:06:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.757+0000 7f8a30600700 1 -- 172.21.15.16:0/1641520194 wait complete. 2024-05-27T07:06:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.758+0000 7f8a30600700 1 Processor -- start 2024-05-27T07:06:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.759+0000 7f8a30600700 1 -- start start 2024-05-27T07:06:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.760+0000 7f8a30600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 0x7f8a281a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a30600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 0x7f8a281a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a30600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a28110290 0x7f8a281a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 0x7f8a281a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 0x7f8a281a80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58724/0 (socket says 172.21.15.16:58724) 2024-05-27T07:06:17.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a2dc00700 1 -- 172.21.15.16:0/1595459281 learned_addr learned my addr 172.21.15.16:0/1595459281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:17.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 0x7f8a281a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.761+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a28110290 0x7f8a281a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.763+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811aa00 con 0x7f8a28110290 2024-05-27T07:06:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.763+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811a880 con 0x7f8a2810f940 2024-05-27T07:06:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.763+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811ab80 con 0x7f8a2810b010 2024-05-27T07:06:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.764+0000 7f8a2d200700 1 -- 172.21.15.16:0/1595459281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 msgr2=0x7f8a281a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.764+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 0x7f8a281a80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.764+0000 7f8a2d200700 1 -- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a28110290 msgr2=0x7f8a281a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.764+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a28110290 0x7f8a281a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.764+0000 7f8a2d200700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a281a5be0 con 0x7f8a2810f940 2024-05-27T07:06:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.765+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 0x7f8a281a11c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005fd0 tx=0x7f8a1c009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.766+0000 7f8a26000700 1 -- 172.21.15.16:0/1595459281 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0090d0 con 0x7f8a2810f940 2024-05-27T07:06:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.766+0000 7f8a26000700 1 -- 172.21.15.16:0/1595459281 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1c0044b0 con 0x7f8a2810f940 2024-05-27T07:06:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.766+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a281a5e70 con 0x7f8a2810f940 2024-05-27T07:06:17.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.767+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281a6400 con 0x7f8a2810f940 2024-05-27T07:06:17.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.767+0000 7f8a26000700 1 -- 172.21.15.16:0/1595459281 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00e5c0 con 0x7f8a2810f940 2024-05-27T07:06:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.780+0000 7f8a26000700 1 -- 172.21.15.16:0/1595459281 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8a1c01b030 con 0x7f8a2810f940 2024-05-27T07:06:17.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.783+0000 7f8a3f0b6b80 1 Processor -- start 2024-05-27T07:06:17.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.783+0000 7f8a3f0b6b80 1 -- start start 2024-05-27T07:06:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.784+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.784+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.785+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.785+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.785+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef867f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35404/0 (socket says 172.21.15.16:35404) 2024-05-27T07:06:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9beeaee40 con 0x55f9bee01e40 2024-05-27T07:06:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9bee9e9e0 con 0x55f9bef86d30 2024-05-27T07:06:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9beea2a20 con 0x55f9bef80db0 2024-05-27T07:06:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.785+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58726/0 (socket says 172.21.15.16:58726) 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a2d200700 1 -- 172.21.15.16:0/3112216668 learned_addr learned my addr 172.21.15.16:0/3112216668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.786+0000 7f8a26000700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 0x7f8a100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.785+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a26000700 1 -- 172.21.15.16:0/1595459281 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a1c0e1670 con 0x7f8a2810f940 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2d200700 1 -- 172.21.15.16:0/3112216668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 msgr2=0x55f9bef80870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2d200700 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef80870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2d200700 1 -- 172.21.15.16:0/3112216668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 msgr2=0x55f9bef867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2d200700 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2d200700 1 -- 172.21.15.16:0/3112216668 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bee67420 con 0x55f9bef80db0 2024-05-27T07:06:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.787+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:17.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.788+0000 7f8a2d200700 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a5860 tx=0x7f8a1c003810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a901dd584ed0bdfc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.788+0000 7f8a1aa00700 1 -- 172.21.15.16:0/3112216668 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0e1070 con 0x55f9bef80db0 2024-05-27T07:06:17.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.788+0000 7f8a1aa00700 1 -- 172.21.15.16:0/3112216668 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1c0a8100 con 0x55f9bef80db0 2024-05-27T07:06:17.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.789+0000 7f8a1aa00700 1 -- 172.21.15.16:0/3112216668 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0e1070 con 0x55f9bef80db0 2024-05-27T07:06:17.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.790+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/3112216668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 msgr2=0x55f9bef811c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.790+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a5860 tx=0x7f8a1c003810 comp rx=0 tx=0).stop 2024-05-27T07:06:17.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.790+0000 7f8a2e600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 0x7f8a100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/3112216668 shutdown_connections 2024-05-27T07:06:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef811c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/3112216668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/3112216668 >> 172.21.15.16:0/3112216668 conn(0x55f9bee61fc0 msgr2=0x55f9bef7b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.791+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/3112216668 shutdown_connections 2024-05-27T07:06:17.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.792+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/3112216668 wait complete. 2024-05-27T07:06:17.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.793+0000 7f8a3f0b6b80 1 -- start start 2024-05-27T07:06:17.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.793+0000 7f8a3f0b6b80 1 Processor -- start 2024-05-27T07:06:17.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.794+0000 7f8a2e600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 0x7f8a100b4f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55f9bef81b30 tx=0x7f8a20009920 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.795+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.795+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.795+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 0x55f9bef8aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.796+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.796+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.796+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58740/0 (socket says 172.21.15.16:58740) 2024-05-27T07:06:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.796+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9beda4fa0 con 0x55f9bef86d30 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9bee3f760 con 0x55f9bee01e40 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9bef8d800 con 0x55f9bef80db0 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a2d200700 1 -- 172.21.15.16:0/1403771661 learned_addr learned my addr 172.21.15.16:0/1403771661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a3f0b6b80 10 client.0 _ll_register_callbacks cb 0x55f9bee48c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.797+0000 7f8a2e600700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 0x55f9bef8aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.798+0000 7f8a2dc00700 1 -- 172.21.15.16:0/1403771661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 msgr2=0x55f9bef8a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.798+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.798+0000 7f8a2dc00700 1 -- 172.21.15.16:0/1403771661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 msgr2=0x55f9bef8aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.798+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 0x55f9bef8aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.798+0000 7f8a2dc00700 1 -- 172.21.15.16:0/1403771661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bedee8d0 con 0x55f9bee01e40 2024-05-27T07:06:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.800+0000 7f8a2e600700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 0x55f9bef8aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:06:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.800+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a14011590 tx=0x7f8a140118a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fc8184ef47ee82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.800+0000 7f8a0f400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:17.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.801+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14014710 con 0x55f9bee01e40 2024-05-27T07:06:17.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.801+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a140148b0 con 0x55f9bee01e40 2024-05-27T07:06:17.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.801+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c0a8100 con 0x55f9bee01e40 2024-05-27T07:06:17.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.801+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a14026050 con 0x55f9bee01e40 2024-05-27T07:06:17.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.802+0000 7f8a3f0b6b80 10 client.35921 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:17.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.802+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9bee6f060 con 0x55f9bee01e40 2024-05-27T07:06:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.803+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1c0a8100 con 0x55f9bee01e40 2024-05-27T07:06:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.803+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8a1401dae0 con 0x55f9bee01e40 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 10 client.35921 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 10 client.35921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 20 client.35921 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bef9a760 0x55f9bef9cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 4 client.35921 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:06:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.804+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9beeb0d90 con 0x55f9bef9a760 2024-05-27T07:06:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.805+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bef9a760 0x55f9bef9cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.808+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bef9a760 0x55f9bef9cb80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=451ebd3fb5bb805a server_cookie=c181696d285e977c in_seq=0 out_seq=0 2024-05-27T07:06:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.808+0000 7f8a0f400700 10 client.35921 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:06:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.809+0000 7f8a0f400700 1 -- 172.21.15.16:0/1403771661 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9beeb0d90 con 0x55f9bef9a760 2024-05-27T07:06:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.809+0000 7f8a0f400700 10 client.35921 handle_command_reply: tid=0 2024-05-27T07:06:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.810+0000 7f8a3f0b6b80 1 client.35921 shutdown 2024-05-27T07:06:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.811+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 msgr2=0x55f9bef8a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.811+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a14011590 tx=0x7f8a140118a0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 shutdown_connections 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bef9a760 0x55f9bef9cb80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a0f400700 0 client.35921 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bef86d30 0x55f9bef8aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bee01e40 0x55f9bef8a470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1403771661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 >> 172.21.15.16:0/1403771661 conn(0x55f9bee61fc0 msgr2=0x55f9bef7b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 shutdown_connections 2024-05-27T07:06:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1403771661 wait complete. 2024-05-27T07:06:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.812+0000 7f8a3f0b6b80 20 client.35921 trim_cache size 0 max 16384 2024-05-27T07:06:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.849+0000 7f8a3f0b6b80 1 Processor -- start 2024-05-27T07:06:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.849+0000 7f8a3f0b6b80 1 -- start start 2024-05-27T07:06:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.849+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.850+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.850+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef96980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58750/0 (socket says 172.21.15.16:58750) 2024-05-27T07:06:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2d200700 1 -- 172.21.15.16:0/1855602246 learned_addr learned my addr 172.21.15.16:0/1855602246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef96980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.851+0000 7f8a2dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef80870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35432/0 (socket says 172.21.15.16:35432) 2024-05-27T07:06:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.852+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0800edf0 con 0x55f9bee01e40 2024-05-27T07:06:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.852+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08010930 con 0x55f9bef86d30 2024-05-27T07:06:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.852+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0800f4c0 con 0x55f9bef80db0 2024-05-27T07:06:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.853+0000 7f8a2d200700 1 -- 172.21.15.16:0/1855602246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 msgr2=0x55f9bef96980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.853+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef96980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.853+0000 7f8a2d200700 1 -- 172.21.15.16:0/1855602246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 msgr2=0x55f9bef80870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.853+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef80870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.853+0000 7f8a2d200700 1 -- 172.21.15.16:0/1855602246 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bef7c710 con 0x55f9bef80db0 2024-05-27T07:06:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.854+0000 7f8a2d200700 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a5860 tx=0x7f8a1c0e1990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d90f24e37fae5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.856+0000 7f8a1a000700 1 -- 172.21.15.16:0/1855602246 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0e1040 con 0x55f9bef80db0 2024-05-27T07:06:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.856+0000 7f8a1a000700 1 -- 172.21.15.16:0/1855602246 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1c0a8cf0 con 0x55f9bef80db0 2024-05-27T07:06:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.856+0000 7f8a1a000700 1 -- 172.21.15.16:0/1855602246 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0e11e0 con 0x55f9bef80db0 2024-05-27T07:06:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.857+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 msgr2=0x55f9bef86000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.857+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a5860 tx=0x7f8a1c0e1990 comp rx=0 tx=0).stop 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.858+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 shutdown_connections 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.858+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef96980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.858+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.858+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/1855602246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef86000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.858+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 >> 172.21.15.16:0/1855602246 conn(0x55f9bee61fc0 msgr2=0x55f9bef84880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.859+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 shutdown_connections 2024-05-27T07:06:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.859+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/1855602246 wait complete. 2024-05-27T07:06:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.859+0000 7f8a3f0b6b80 1 -- start start 2024-05-27T07:06:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.860+0000 7f8a3f0b6b80 1 Processor -- start 2024-05-27T07:06:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.863+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef93280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08010260 con 0x55f9bee01e40 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0800fb90 con 0x55f9bef86d30 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0800cd90 con 0x55f9bef80db0 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a3f0b6b80 10 client.0 _ll_register_callbacks cb 0x55f9bee48c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.864+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.865+0000 7f8a2e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:47802/0 (socket says 172.21.15.16:47802) 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.865+0000 7f8a2e600700 1 -- 172.21.15.16:0/855995759 learned_addr learned my addr 172.21.15.16:0/855995759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.865+0000 7f8a2e600700 1 -- 172.21.15.16:0/855995759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 msgr2=0x55f9bef8b940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.865+0000 7f8a2e600700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.865+0000 7f8a2e600700 1 -- 172.21.15.16:0/855995759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 msgr2=0x55f9bef93280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:06:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.866+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef93280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.866+0000 7f8a2e600700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef93280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.866+0000 7f8a2e600700 1 -- 172.21.15.16:0/855995759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bef8b260 con 0x55f9bef86d30 2024-05-27T07:06:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.867+0000 7f8a2e600700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000be90 tx=0x7f8a2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18936b128aabfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.867+0000 7f8a0f400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.867+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20007c30 con 0x55f9bef86d30 2024-05-27T07:06:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.867+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f8a200050c0 con 0x55f9bef86d30 2024-05-27T07:06:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.868+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c0a8cf0 con 0x55f9bef86d30 2024-05-27T07:06:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.869+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a200052c0 con 0x55f9bef86d30 2024-05-27T07:06:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.869+0000 7f8a3f0b6b80 10 client.35927 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.869+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9bf004e60 con 0x55f9bef86d30 2024-05-27T07:06:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1c0a8cf0 con 0x55f9bef86d30 2024-05-27T07:06:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8a2003a020 con 0x55f9bef86d30 2024-05-27T07:06:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 10 client.35927 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 10 client.35927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:06:17.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 20 client.35927 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:17.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bf005620 0x55f9bf007a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:17.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 4 client.35927 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:06:17.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9beeaa570 con 0x55f9bf005620 2024-05-27T07:06:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.871+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bf005620 0x55f9bf007a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.872+0000 7f8a2dc00700 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bf005620 0x55f9bf007a40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12ad452690790570 server_cookie=ac3fb505e40f239c in_seq=0 out_seq=0 2024-05-27T07:06:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.872+0000 7f8a0f400700 10 client.35927 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:06:17.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.873+0000 7f8a0f400700 1 -- 172.21.15.16:0/855995759 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9beeaa570 con 0x55f9bf005620 2024-05-27T07:06:17.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.873+0000 7f8a0f400700 10 client.35927 handle_command_reply: tid=0 2024-05-27T07:06:17.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.873+0000 7f8a3f0b6b80 1 client.35927 shutdown 2024-05-27T07:06:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.873+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 msgr2=0x55f9bef8be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.873+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000be90 tx=0x7f8a2000b330 comp rx=0 tx=0).stop 2024-05-27T07:06:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.875+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 shutdown_connections 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.875+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f9bf005620 0x55f9bf007a40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.875+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9bef86d30 0x55f9bef8be80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.875+0000 7f8a0f400700 0 client.35927 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.875+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bee01e40 0x55f9bef93280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.876+0000 7f8a3f0b6b80 1 --2- 172.21.15.16:0/855995759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9bef80db0 0x55f9bef8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.876+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 >> 172.21.15.16:0/855995759 conn(0x55f9bee61fc0 msgr2=0x55f9bef84880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.876+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 shutdown_connections 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.876+0000 7f8a3f0b6b80 1 -- 172.21.15.16:0/855995759 wait complete. 2024-05-27T07:06:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.876+0000 7f8a3f0b6b80 20 client.35927 trim_cache size 0 max 16384 2024-05-27T07:06:17.870 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:06:17.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 msgr2=0x7f8a100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 0x7f8a100b4f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55f9bef81b30 tx=0x7f8a20009920 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 msgr2=0x7f8a281a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 0x7f8a281a11c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005fd0 tx=0x7f8a1c009ce0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 shutdown_connections 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8a100b2a70 0x7f8a100b4f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a28110290 0x7f8a281a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2810b010 0x7f8a281a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 --2- 172.21.15.16:0/1595459281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a2810f940 0x7f8a281a11c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.878+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 >> 172.21.15.16:0/1595459281 conn(0x7f8a2806f980 msgr2=0x7f8a2810f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.879+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 shutdown_connections 2024-05-27T07:06:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:17.879+0000 7f8a30600700 1 -- 172.21.15.16:0/1595459281 wait complete. 2024-05-27T07:06:18.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:06:19.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:18 smithi016 ceph-mon[31415]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 30 MiB/s wr, 490 op/s 2024-05-27T07:06:19.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:18 smithi164 ceph-mon[35327]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 30 MiB/s wr, 490 op/s 2024-05-27T07:06:19.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:18 smithi069 ceph-mon[35276]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 30 MiB/s wr, 490 op/s 2024-05-27T07:06:19.873 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:21.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:20 smithi164 ceph-mon[35327]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 34 MiB/s wr, 547 op/s 2024-05-27T07:06:21.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:20 smithi016 ceph-mon[31415]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 34 MiB/s wr, 547 op/s 2024-05-27T07:06:21.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:20 smithi069 ceph-mon[35276]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 34 MiB/s wr, 547 op/s 2024-05-27T07:06:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.910+0000 7fe00f800700 1 -- 172.21.15.16:0/1036078742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103b90 msgr2=0x7fe0081080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.910+0000 7fe00f800700 1 --2- 172.21.15.16:0/1036078742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103b90 0x7fe0081080c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009990 tx=0x7fdff4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.912+0000 7fe00f800700 1 -- 172.21.15.16:0/1036078742 shutdown_connections 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.912+0000 7fe00f800700 1 --2- 172.21.15.16:0/1036078742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 0x7fe008108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.912+0000 7fe00f800700 1 --2- 172.21.15.16:0/1036078742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103240 0x7fe008103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.912+0000 7fe00f800700 1 --2- 172.21.15.16:0/1036078742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103b90 0x7fe0081080c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.912+0000 7fe00f800700 1 -- 172.21.15.16:0/1036078742 >> 172.21.15.16:0/1036078742 conn(0x7fe0080fedb0 msgr2=0x7fe0081011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.913+0000 7fe00f800700 1 -- 172.21.15.16:0/1036078742 shutdown_connections 2024-05-27T07:06:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.913+0000 7fe00f800700 1 -- 172.21.15.16:0/1036078742 wait complete. 2024-05-27T07:06:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.915+0000 7fe00f800700 1 Processor -- start 2024-05-27T07:06:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.915+0000 7fe00f800700 1 -- start start 2024-05-27T07:06:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 0x7fe0081a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 0x7fe0081a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe00811a960 con 0x7fe008108690 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe00811a7e0 con 0x7fe008103b90 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00f800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe00811aae0 con 0x7fe008103240 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00ce00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00ce00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:46774/0 (socket says 172.21.15.16:46774) 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.916+0000 7fe00ce00700 1 -- 172.21.15.16:0/241122650 learned_addr learned my addr 172.21.15.16:0/241122650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe00d800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 0x7fe0081a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe00ce00700 1 -- 172.21.15.16:0/241122650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 msgr2=0x7fe0081a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe007e00700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 0x7fe0081a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe00ce00700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 0x7fe0081a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe00ce00700 1 -- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 msgr2=0x7fe0081a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.917+0000 7fe00ce00700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 0x7fe0081a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.918+0000 7fe00ce00700 1 -- 172.21.15.16:0/241122650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081a6220 con 0x7fe008103240 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.918+0000 7fe00ce00700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0060b0 tx=0x7fdffc007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.919+0000 7fe007e00700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 0x7fe0081a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.919+0000 7fe005600700 1 -- 172.21.15.16:0/241122650 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc01a070 con 0x7fe008103240 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.919+0000 7fe005600700 1 -- 172.21.15.16:0/241122650 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdffc00de20 con 0x7fe008103240 2024-05-27T07:06:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.919+0000 7fe005600700 1 -- 172.21.15.16:0/241122650 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc014690 con 0x7fe008103240 2024-05-27T07:06:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.919+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081a6510 con 0x7fe008103240 2024-05-27T07:06:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.920+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00806d1d0 con 0x7fe008103240 2024-05-27T07:06:22.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.926+0000 7fe005600700 1 -- 172.21.15.16:0/241122650 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdffc01d070 con 0x7fe008103240 2024-05-27T07:06:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.927+0000 7fe01e39db80 1 Processor -- start 2024-05-27T07:06:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.928+0000 7fe01e39db80 1 -- start start 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5654865fbe40 con 0x56548654ee40 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5654865eb9e0 con 0x5654866cddb0 2024-05-27T07:06:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.930+0000 7fe01e39db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654865efa20 con 0x5654866d3d30 2024-05-27T07:06:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41296/0 (socket says 172.21.15.16:41296) 2024-05-27T07:06:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 -- 172.21.15.16:0/3898150804 learned_addr learned my addr 172.21.15.16:0/3898150804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 -- 172.21.15.16:0/3898150804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 msgr2=0x5654866cd870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.931+0000 7fe007e00700 1 -- 172.21.15.16:0/3898150804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 msgr2=0x5654866d37f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.932+0000 7fe00d800700 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.932+0000 7fe007e00700 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.932+0000 7fe007e00700 1 -- 172.21.15.16:0/3898150804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654865b4420 con 0x5654866cddb0 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.932+0000 7fe00d800700 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.933+0000 7fe007e00700 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005850 tx=0x7fdff40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf92c8a1fd1bb686 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.933+0000 7fe005600700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 0x7fdff00b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.934+0000 7fe005600700 1 -- 172.21.15.16:0/241122650 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fdffc0e2200 con 0x7fe008103240 2024-05-27T07:06:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.935+0000 7fe00d800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 0x7fdff00b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.936+0000 7fdffa000700 1 -- 172.21.15.16:0/3898150804 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4025070 con 0x5654866cddb0 2024-05-27T07:06:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.937+0000 7fdffa000700 1 -- 172.21.15.16:0/3898150804 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff4005500 con 0x5654866cddb0 2024-05-27T07:06:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.937+0000 7fdffa000700 1 -- 172.21.15.16:0/3898150804 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ba30 con 0x5654866cddb0 2024-05-27T07:06:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.937+0000 7fe00d800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 0x7fdff00b5110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe000005e90 tx=0x7fe000005d40 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.940+0000 7fe01e39db80 1 -- 172.21.15.16:0/3898150804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 msgr2=0x5654866ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.941+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005850 tx=0x7fdff40096a0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.942+0000 7fe01e39db80 1 -- 172.21.15.16:0/3898150804 shutdown_connections 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.942+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866ce1c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.942+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.942+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3898150804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.942+0000 7fe01e39db80 1 -- 172.21.15.16:0/3898150804 >> 172.21.15.16:0/3898150804 conn(0x5654865aefc0 msgr2=0x5654866c88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.943+0000 7fe01e39db80 1 -- 172.21.15.16:0/3898150804 shutdown_connections 2024-05-27T07:06:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.943+0000 7fe01e39db80 1 -- 172.21.15.16:0/3898150804 wait complete. 2024-05-27T07:06:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.944+0000 7fe01e39db80 1 -- start start 2024-05-27T07:06:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.944+0000 7fe01e39db80 1 Processor -- start 2024-05-27T07:06:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56548654ee40 0x5654866d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x5654866d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5654864f1fa0 con 0x5654866d3d30 2024-05-27T07:06:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56548658c760 con 0x5654866cddb0 2024-05-27T07:06:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654866da7a0 con 0x56548654ee40 2024-05-27T07:06:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.945+0000 7fe01e39db80 10 client.0 _ll_register_callbacks cb 0x565486595c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.947+0000 7fe00ce00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56548654ee40 0x5654866d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.947+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.947+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41298/0 (socket says 172.21.15.16:41298) 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.947+0000 7fe007e00700 1 -- 172.21.15.16:0/3719521947 learned_addr learned my addr 172.21.15.16:0/3719521947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.948+0000 7fe007e00700 1 -- 172.21.15.16:0/3719521947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56548654ee40 msgr2=0x5654866d7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.948+0000 7fe007e00700 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56548654ee40 0x5654866d7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.948+0000 7fe007e00700 1 -- 172.21.15.16:0/3719521947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 msgr2=0x5654866d8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.948+0000 7fe007e00700 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x5654866d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.948+0000 7fe007e00700 1 -- 172.21.15.16:0/3719521947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548653b8d0 con 0x5654866cddb0 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.949+0000 7fdfeea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.949+0000 7fe007e00700 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdff4023040 tx=0x7fdff4009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af607be2114b3669 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.949+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4030060 con 0x5654866cddb0 2024-05-27T07:06:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.949+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fdff400b9e0 con 0x5654866cddb0 2024-05-27T07:06:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.950+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff4005500 con 0x5654866cddb0 2024-05-27T07:06:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.950+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdff4025070 con 0x5654866cddb0 2024-05-27T07:06:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.951+0000 7fe01e39db80 10 client.35936 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.951+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654865bc060 con 0x5654866cddb0 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.953+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff4005500 con 0x5654866cddb0 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.953+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdff400fc00 con 0x5654866cddb0 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.954+0000 7fe01e39db80 10 client.35936 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.955+0000 7fe01e39db80 10 client.35936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.955+0000 7fe01e39db80 20 client.35936 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.955+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654866e7530 0x5654866e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.955+0000 7fe01e39db80 4 client.35936 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:06:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.955+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654865fdd90 con 0x5654866e7530 2024-05-27T07:06:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.956+0000 7fe00ce00700 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654866e7530 0x5654866e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.956+0000 7fe00ce00700 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654866e7530 0x5654866e9950 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd798db078a81ca8 server_cookie=263f2998e08bba1c in_seq=0 out_seq=0 2024-05-27T07:06:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.957+0000 7fdfeea00700 10 client.35936 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.958+0000 7fdfeea00700 1 -- 172.21.15.16:0/3719521947 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654865fdd90 con 0x5654866e7530 2024-05-27T07:06:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.958+0000 7fdfeea00700 10 client.35936 handle_command_reply: tid=0 2024-05-27T07:06:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.958+0000 7fe01e39db80 1 client.35936 shutdown 2024-05-27T07:06:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 msgr2=0x5654866d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdff4023040 tx=0x7fdff4009710 comp rx=0 tx=0).stop 2024-05-27T07:06:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 shutdown_connections 2024-05-27T07:06:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5654866e7530 0x5654866e9950 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x5654866d8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56548654ee40 0x5654866d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3719521947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d7ad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.959+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 >> 172.21.15.16:0/3719521947 conn(0x5654865aefc0 msgr2=0x5654866c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.960+0000 7fdfeea00700 0 client.35936 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.960+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 shutdown_connections 2024-05-27T07:06:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.960+0000 7fe01e39db80 1 -- 172.21.15.16:0/3719521947 wait complete. 2024-05-27T07:06:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.961+0000 7fe01e39db80 20 client.35936 trim_cache size 0 max 16384 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.996+0000 7fe01e39db80 1 Processor -- start 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.996+0000 7fe01e39db80 1 -- start start 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081bf200 con 0x56548654ee40 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081c0d40 con 0x5654866cddb0 2024-05-27T07:06:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.997+0000 7fe01e39db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081bf8d0 con 0x5654866d3d30 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.998+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.998+0000 7fe007e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41312/0 (socket says 172.21.15.16:41312) 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.998+0000 7fe007e00700 1 -- 172.21.15.16:0/2473924816 learned_addr learned my addr 172.21.15.16:0/2473924816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.998+0000 7fe00d800700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.998+0000 7fe00ce00700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe007e00700 1 -- 172.21.15.16:0/2473924816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 msgr2=0x5654866d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe007e00700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe007e00700 1 -- 172.21.15.16:0/2473924816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 msgr2=0x5654866cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe007e00700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe007e00700 1 -- 172.21.15.16:0/2473924816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654866d8380 con 0x5654866cddb0 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:22.999+0000 7fe00ce00700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.000+0000 7fe007e00700 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdff40205e0 tx=0x7fdff40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e2a7be8f5cb9bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.000+0000 7fdff9600700 1 -- 172.21.15.16:0/2473924816 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4030070 con 0x5654866cddb0 2024-05-27T07:06:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.000+0000 7fdff9600700 1 -- 172.21.15.16:0/2473924816 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff400be70 con 0x5654866cddb0 2024-05-27T07:06:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.000+0000 7fdff9600700 1 -- 172.21.15.16:0/2473924816 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4030070 con 0x5654866cddb0 2024-05-27T07:06:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.002+0000 7fe01e39db80 1 -- 172.21.15.16:0/2473924816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 msgr2=0x5654866d9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.002+0000 7fe01e39db80 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdff40205e0 tx=0x7fdff40096a0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.002+0000 7fe01e39db80 1 -- 172.21.15.16:0/2473924816 shutdown_connections 2024-05-27T07:06:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.003+0000 7fe01e39db80 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866d3d30 0x5654866d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.003+0000 7fe01e39db80 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56548654ee40 0x5654866cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.003+0000 7fe01e39db80 1 --2- 172.21.15.16:0/2473924816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654866cddb0 0x5654866d9a30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.003+0000 7fe01e39db80 1 -- 172.21.15.16:0/2473924816 >> 172.21.15.16:0/2473924816 conn(0x5654865aefc0 msgr2=0x5654866d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.004+0000 7fe01e39db80 1 -- 172.21.15.16:0/2473924816 shutdown_connections 2024-05-27T07:06:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.004+0000 7fe01e39db80 1 -- 172.21.15.16:0/2473924816 wait complete. 2024-05-27T07:06:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.005+0000 7fe01e39db80 1 -- start start 2024-05-27T07:06:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.006+0000 7fe01e39db80 1 Processor -- start 2024-05-27T07:06:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 0x5654866ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x565486749dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081c0670 con 0x5654866d3d30 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081bffa0 con 0x56548654ee40 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe008073d40 con 0x5654866cddb0 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.007+0000 7fe01e39db80 10 client.0 _ll_register_callbacks cb 0x565486595c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.008+0000 7fe00ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.008+0000 7fe00ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:41324/0 (socket says 172.21.15.16:41324) 2024-05-27T07:06:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.008+0000 7fe00ce00700 1 -- 172.21.15.16:0/3694089742 learned_addr learned my addr 172.21.15.16:0/3694089742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.008+0000 7fe007e00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 0x5654866ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.009+0000 7fe00ce00700 1 -- 172.21.15.16:0/3694089742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 msgr2=0x5654866ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.009+0000 7fe00ce00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 0x5654866ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.009+0000 7fe00ce00700 1 -- 172.21.15.16:0/3694089742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 msgr2=0x565486749dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.009+0000 7fe00ce00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x565486749dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.009+0000 7fe00ce00700 1 -- 172.21.15.16:0/3694089742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548674a3c0 con 0x56548654ee40 2024-05-27T07:06:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.010+0000 7fe007e00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 0x5654866ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.010+0000 7fe00ce00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdffc017000 tx=0x7fdffc015d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb67dc1c51ab0329 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.010+0000 7fdfeea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.011+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0b1f50 con 0x56548654ee40 2024-05-27T07:06:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.011+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fdffc0df7e0 con 0x56548654ee40 2024-05-27T07:06:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.012+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff400be70 con 0x56548654ee40 2024-05-27T07:06:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.012+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdffc0df9e0 con 0x56548654ee40 2024-05-27T07:06:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.013+0000 7fe01e39db80 10 client.35942 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.013+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565486751720 con 0x56548654ee40 2024-05-27T07:06:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.015+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff400be70 con 0x56548654ee40 2024-05-27T07:06:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.015+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdffc0b1ac0 con 0x56548654ee40 2024-05-27T07:06:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 10 client.35942 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 10 client.35942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 20 client.35942 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565486751e50 0x565486754270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 4 client.35942 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:06:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.016+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654865f7570 con 0x565486751e50 2024-05-27T07:06:23.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.017+0000 7fe007e00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565486751e50 0x565486754270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.019+0000 7fdfeea00700 10 client.35942 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.019+0000 7fe007e00700 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565486751e50 0x565486754270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e0059f9da5f03a server_cookie=2daaba7b90c997f in_seq=0 out_seq=0 2024-05-27T07:06:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.019+0000 7fdfeea00700 1 -- 172.21.15.16:0/3694089742 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654865f7570 con 0x565486751e50 2024-05-27T07:06:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.019+0000 7fdfeea00700 10 client.35942 handle_command_reply: tid=0 2024-05-27T07:06:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.019+0000 7fe01e39db80 1 client.35942 shutdown 2024-05-27T07:06:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 msgr2=0x5654866ea130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdffc017000 tx=0x7fdffc015d60 comp rx=0 tx=0).stop 2024-05-27T07:06:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 shutdown_connections 2024-05-27T07:06:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565486751e50 0x565486754270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fdfeea00700 0 client.35942 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654866d3d30 0x565486749dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56548654ee40 0x5654866ea130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 --2- 172.21.15.16:0/3694089742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654866cddb0 0x5654866ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.020+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 >> 172.21.15.16:0/3694089742 conn(0x5654865aefc0 msgr2=0x5654866d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.021+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 shutdown_connections 2024-05-27T07:06:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.021+0000 7fe01e39db80 1 -- 172.21.15.16:0/3694089742 wait complete. 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.021+0000 7fe01e39db80 20 client.35942 trim_cache size 0 max 16384 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.022+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 msgr2=0x7fdff00b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.022+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 0x7fdff00b5110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe000005e90 tx=0x7fe000005d40 comp rx=0 tx=0).stop 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.022+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 msgr2=0x7fe0081a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.022+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0060b0 tx=0x7fdffc007f90 comp rx=0 tx=0).stop 2024-05-27T07:06:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 shutdown_connections 2024-05-27T07:06:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdff00b2c00 0x7fdff00b5110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe008108690 0x7fe0081a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008103240 0x7fe0081a1350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 --2- 172.21.15.16:0/241122650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe008103b90 0x7fe0081a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 >> 172.21.15.16:0/241122650 conn(0x7fe0080fedb0 msgr2=0x7fe008105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.024+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 shutdown_connections 2024-05-27T07:06:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:23.026+0000 7fe00f800700 1 -- 172.21.15.16:0/241122650 wait complete. 2024-05-27T07:06:23.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:22 smithi164 ceph-mon[35327]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 38 MiB/s wr, 619 op/s 2024-05-27T07:06:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:22 smithi016 ceph-mon[31415]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 38 MiB/s wr, 619 op/s 2024-05-27T07:06:23.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:22 smithi069 ceph-mon[35276]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 38 MiB/s wr, 619 op/s 2024-05-27T07:06:25.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:24 smithi164 ceph-mon[35327]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:06:25.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:24 smithi016 ceph-mon[31415]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:06:25.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:24 smithi069 ceph-mon[35276]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:06:25.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:06:25.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:06:26.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:25 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:06:26.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:25 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:06:26.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:25 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:06:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:27.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:26 smithi016 ceph-mon[31415]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 699 op/s 2024-05-27T07:06:27.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:26 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:27.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:26 smithi164 ceph-mon[35327]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 699 op/s 2024-05-27T07:06:27.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:26 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:27.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:26 smithi069 ceph-mon[35276]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 699 op/s 2024-05-27T07:06:27.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:27 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:28 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:28 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:28 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:28 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:28 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:28 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:28 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:28 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:28.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:28 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:29.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:29 smithi164 ceph-mon[35327]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 31 MiB/s wr, 508 op/s 2024-05-27T07:06:29.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:29 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:06:29.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:29 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:06:29.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:29 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:29 smithi016 ceph-mon[31415]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 31 MiB/s wr, 508 op/s 2024-05-27T07:06:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:29 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:06:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:29 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:06:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:29 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:29.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:29 smithi069 ceph-mon[35276]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 31 MiB/s wr, 508 op/s 2024-05-27T07:06:29.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:29 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:06:29.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:29 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:06:29.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:29 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:06:31.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:31 smithi164 ceph-mon[35327]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 35 MiB/s wr, 561 op/s 2024-05-27T07:06:31.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:31 smithi016 ceph-mon[31415]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 35 MiB/s wr, 561 op/s 2024-05-27T07:06:31.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:31 smithi069 ceph-mon[35276]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 35 MiB/s wr, 561 op/s 2024-05-27T07:06:33.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:33 smithi164 ceph-mon[35327]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 620 op/s 2024-05-27T07:06:33.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:33 smithi016 ceph-mon[31415]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 620 op/s 2024-05-27T07:06:33.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:33 smithi069 ceph-mon[35276]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 MiB/s wr, 620 op/s 2024-05-27T07:06:35.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:35 smithi164 ceph-mon[35327]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:35.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:35 smithi016 ceph-mon[31415]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:35.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:35 smithi069 ceph-mon[35276]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 -- 172.21.15.16:0/2326409741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0109840 msgr2=0x7ff2f01111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 --2- 172.21.15.16:0/2326409741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0109840 0x7ff2f01111a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009990 tx=0x7ff2e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:06:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 -- 172.21.15.16:0/2326409741 shutdown_connections 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 --2- 172.21.15.16:0/2326409741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0109840 0x7ff2f01111a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 --2- 172.21.15.16:0/2326409741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f01045c0 0x7ff2f01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 --2- 172.21.15.16:0/2326409741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0108ef0 0x7ff2f0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.005+0000 7ff2f8600700 1 -- 172.21.15.16:0/2326409741 >> 172.21.15.16:0/2326409741 conn(0x7ff2f00ffe90 msgr2=0x7ff2f01022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.006+0000 7ff2f8600700 1 -- 172.21.15.16:0/2326409741 shutdown_connections 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.006+0000 7ff2f8600700 1 -- 172.21.15.16:0/2326409741 wait complete. 2024-05-27T07:06:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.007+0000 7ff2f8600700 1 Processor -- start 2024-05-27T07:06:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 -- start start 2024-05-27T07:06:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f01045c0 0x7ff2f01a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0109840 0x7ff2f01ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0114370 con 0x7ff2f01045c0 2024-05-27T07:06:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01141f0 con 0x7ff2f0108ef0 2024-05-27T07:06:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.008+0000 7ff2f8600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01144f0 con 0x7ff2f0109840 2024-05-27T07:06:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.009+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.009+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51380/0 (socket says 172.21.15.16:51380) 2024-05-27T07:06:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.009+0000 7ff2f5200700 1 -- 172.21.15.16:0/847736702 learned_addr learned my addr 172.21.15.16:0/847736702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.010+0000 7ff2f6600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0109840 0x7ff2f01ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.011+0000 7ff2f5200700 1 -- 172.21.15.16:0/847736702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0109840 msgr2=0x7ff2f01ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.011+0000 7ff2f5200700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0109840 0x7ff2f01ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.011+0000 7ff2f5200700 1 -- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f01045c0 msgr2=0x7ff2f01a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.011+0000 7ff2f5200700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f01045c0 0x7ff2f01a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.011+0000 7ff2f5200700 1 -- 172.21.15.16:0/847736702 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f0199840 con 0x7ff2f0108ef0 2024-05-27T07:06:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.012+0000 7ff2f5200700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d980 tx=0x7ff2dc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.012+0000 7ff2ee000700 1 -- 172.21.15.16:0/847736702 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc017070 con 0x7ff2f0108ef0 2024-05-27T07:06:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.013+0000 7ff2ee000700 1 -- 172.21.15.16:0/847736702 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2dc009dc0 con 0x7ff2f0108ef0 2024-05-27T07:06:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.013+0000 7ff2ee000700 1 -- 172.21.15.16:0/847736702 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00f3c0 con 0x7ff2f0108ef0 2024-05-27T07:06:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.013+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f0199b30 con 0x7ff2f0108ef0 2024-05-27T07:06:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.014+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f019a020 con 0x7ff2f0108ef0 2024-05-27T07:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.029+0000 7ff307115b80 1 Processor -- start 2024-05-27T07:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.030+0000 7ff307115b80 1 -- start start 2024-05-27T07:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 0x55aae5f288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f29240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aae5e56ec0 con 0x55aae5f28e30 2024-05-27T07:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aae5e46a60 con 0x55aae5f2edb0 2024-05-27T07:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.031+0000 7ff307115b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aae5e4aaa0 con 0x55aae5da9ec0 2024-05-27T07:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.033+0000 7ff2f6600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.033+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 0x55aae5f288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.033+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 0x55aae5f288f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51396/0 (socket says 172.21.15.16:51396) 2024-05-27T07:06:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.033+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2899314596 learned_addr learned my addr 172.21.15.16:0/2899314596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.033+0000 7ff2f6600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59790/0 (socket says 172.21.15.16:59790) 2024-05-27T07:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.034+0000 7ff2f6600700 1 -- 172.21.15.16:0/2899314596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 msgr2=0x55aae5f288f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.034+0000 7ff2f6600700 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 0x55aae5f288f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.034+0000 7ff2f6600700 1 -- 172.21.15.16:0/2899314596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 msgr2=0x55aae5f29240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.034+0000 7ff2f6600700 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f29240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.034+0000 7ff2f6600700 1 -- 172.21.15.16:0/2899314596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae5e0f4a0 con 0x55aae5da9ec0 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.035+0000 7ff2ee000700 1 -- 172.21.15.16:0/847736702 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff2dc009930 con 0x7ff2f0108ef0 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.038+0000 7ff2ee000700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 0x7ff2d80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.038+0000 7ff2ee000700 1 -- 172.21.15.16:0/847736702 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff2dc0e2450 con 0x7ff2f0108ef0 2024-05-27T07:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.039+0000 7ff2f6600700 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40098e0 tx=0x7ff2e4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744bebba779452f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.040+0000 7ff2e2a00700 1 -- 172.21.15.16:0/2899314596 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4005390 con 0x55aae5da9ec0 2024-05-27T07:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.040+0000 7ff2e2a00700 1 -- 172.21.15.16:0/2899314596 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e4005530 con 0x55aae5da9ec0 2024-05-27T07:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.040+0000 7ff2e2a00700 1 -- 172.21.15.16:0/2899314596 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4005390 con 0x55aae5da9ec0 2024-05-27T07:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.041+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 0x7ff2d80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.043+0000 7ff307115b80 1 -- 172.21.15.16:0/2899314596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 msgr2=0x55aae5f2e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.043+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 0x7ff2d80b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009510 tx=0x7ff2e8009040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.043+0000 7ff307115b80 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40098e0 tx=0x7ff2e4005f00 comp rx=0 tx=0).stop 2024-05-27T07:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 -- 172.21.15.16:0/2899314596 shutdown_connections 2024-05-27T07:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f29240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f2e870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 --2- 172.21.15.16:0/2899314596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f2edb0 0x55aae5f288f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 -- 172.21.15.16:0/2899314596 >> 172.21.15.16:0/2899314596 conn(0x55aae5e0a040 msgr2=0x55aae5f23950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 -- 172.21.15.16:0/2899314596 shutdown_connections 2024-05-27T07:06:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.046+0000 7ff307115b80 1 -- 172.21.15.16:0/2899314596 wait complete. 2024-05-27T07:06:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.047+0000 7ff307115b80 1 -- start start 2024-05-27T07:06:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.047+0000 7ff307115b80 1 Processor -- start 2024-05-27T07:06:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.049+0000 7ff307115b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.049+0000 7ff307115b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.050+0000 7ff307115b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f33070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.050+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.050+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51408/0 (socket says 172.21.15.16:51408) 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59800/0 (socket says 172.21.15.16:59800) 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2504192156 learned_addr learned my addr 172.21.15.16:0/2504192156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aae5d4d020 con 0x55aae5f2edb0 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aae5de77e0 con 0x55aae5da9ec0 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aae5f357c0 con 0x55aae5f28e30 2024-05-27T07:06:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.051+0000 7ff307115b80 10 client.0 _ll_register_callbacks cb 0x55aae5df0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f6600700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f33070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2504192156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 msgr2=0x55aae5f32b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2504192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 msgr2=0x55aae5f33070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f33070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae5d96950 con 0x55aae5da9ec0 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff2f5200700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.052+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e4005530 con 0x55aae5da9ec0 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.053+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8015d80 tx=0x7ff2e8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b9629d250d3051 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.053+0000 7ff2d7400700 10 client.35954 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.054+0000 7ff2d7400700 1 -- 172.21.15.16:0/2504192156 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004db0 con 0x55aae5da9ec0 2024-05-27T07:06:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.054+0000 7ff2d7400700 1 -- 172.21.15.16:0/2504192156 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff2e8016b90 con 0x55aae5da9ec0 2024-05-27T07:06:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.054+0000 7ff2d7400700 1 -- 172.21.15.16:0/2504192156 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2e8016df0 con 0x55aae5da9ec0 2024-05-27T07:06:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.055+0000 7ff307115b80 10 client.35954 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.055+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55aae5e170e0 con 0x55aae5da9ec0 2024-05-27T07:06:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.057+0000 7ff2d7400700 1 -- 172.21.15.16:0/2504192156 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff2e8021400 con 0x55aae5da9ec0 2024-05-27T07:06:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.058+0000 7ff307115b80 10 client.35954 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.058+0000 7ff307115b80 10 client.35954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.058+0000 7ff307115b80 20 client.35954 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.060+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5f42910 0x55aae5f44e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.060+0000 7ff307115b80 4 client.35954 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:06:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.060+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae5e58e10 con 0x55aae5f42910 2024-05-27T07:06:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.060+0000 7ff2f5200700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5f42910 0x55aae5f44e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.062+0000 7ff2d7400700 10 client.35954 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.062+0000 7ff2f5200700 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5f42910 0x55aae5f44e00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a5a42004de291b server_cookie=c06d8a482d7026af in_seq=0 out_seq=0 2024-05-27T07:06:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.063+0000 7ff2d7400700 1 -- 172.21.15.16:0/2504192156 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aae5e58e10 con 0x55aae5f42910 2024-05-27T07:06:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.063+0000 7ff2d7400700 10 client.35954 handle_command_reply: tid=0 2024-05-27T07:06:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.063+0000 7ff307115b80 1 client.35954 shutdown 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 msgr2=0x55aae5f325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8015d80 tx=0x7ff2e8009620 comp rx=0 tx=0).stop 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 shutdown_connections 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5f42910 0x55aae5f44e00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f33070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f325f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 --2- 172.21.15.16:0/2504192156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f28e30 0x55aae5f32b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.065+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 >> 172.21.15.16:0/2504192156 conn(0x55aae5e0a040 msgr2=0x55aae5f23600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.066+0000 7ff2d7400700 0 client.35954 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.066+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 shutdown_connections 2024-05-27T07:06:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.067+0000 7ff307115b80 1 -- 172.21.15.16:0/2504192156 wait complete. 2024-05-27T07:06:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.067+0000 7ff307115b80 20 client.35954 trim_cache size 0 max 16384 2024-05-27T07:06:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.125+0000 7ff307115b80 1 Processor -- start 2024-05-27T07:06:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.125+0000 7ff307115b80 1 -- start start 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f34a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f2edb0 0x55aae5f32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01bf350 con 0x55aae5f28e30 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01c0e90 con 0x55aae5da9ec0 2024-05-27T07:06:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.126+0000 7ff307115b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01bfa20 con 0x55aae5f2edb0 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51424/0 (socket says 172.21.15.16:51424) 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2827380596 learned_addr learned my addr 172.21.15.16:0/2827380596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f6600700 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f2edb0 0x55aae5f32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2827380596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f2edb0 msgr2=0x55aae5f32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f2edb0 0x55aae5f32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2827380596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 msgr2=0x55aae5f34a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f34a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.127+0000 7ff2f5c00700 1 -- 172.21.15.16:0/2827380596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae5f25390 con 0x55aae5da9ec0 2024-05-27T07:06:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.128+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8015f50 tx=0x7ff2e801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26014f4e27f680d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.129+0000 7ff2e2000700 1 -- 172.21.15.16:0/2827380596 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004bb0 con 0x55aae5da9ec0 2024-05-27T07:06:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.129+0000 7ff2e2000700 1 -- 172.21.15.16:0/2827380596 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e4005530 con 0x55aae5da9ec0 2024-05-27T07:06:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.129+0000 7ff2e2000700 1 -- 172.21.15.16:0/2827380596 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004bb0 con 0x55aae5da9ec0 2024-05-27T07:06:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 -- 172.21.15.16:0/2827380596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 msgr2=0x55aae5f288f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8015f50 tx=0x7ff2e801f000 comp rx=0 tx=0).stop 2024-05-27T07:06:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 -- 172.21.15.16:0/2827380596 shutdown_connections 2024-05-27T07:06:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5f2edb0 0x55aae5f32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5da9ec0 0x55aae5f288f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 --2- 172.21.15.16:0/2827380596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f28e30 0x55aae5f34a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.131+0000 7ff307115b80 1 -- 172.21.15.16:0/2827380596 >> 172.21.15.16:0/2827380596 conn(0x55aae5e0a040 msgr2=0x55aae5f2c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.132+0000 7ff307115b80 1 -- 172.21.15.16:0/2827380596 shutdown_connections 2024-05-27T07:06:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.132+0000 7ff307115b80 1 -- 172.21.15.16:0/2827380596 wait complete. 2024-05-27T07:06:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.132+0000 7ff307115b80 1 -- start start 2024-05-27T07:06:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.133+0000 7ff307115b80 1 Processor -- start 2024-05-27T07:06:36.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff307115b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff307115b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff307115b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.134+0000 7ff2f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51428/0 (socket says 172.21.15.16:51428) 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.135+0000 7ff2f5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f47d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59810/0 (socket says 172.21.15.16:59810) 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.135+0000 7ff2f5200700 1 -- 172.21.15.16:0/1497557244 learned_addr learned my addr 172.21.15.16:0/1497557244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.135+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01c07c0 con 0x55aae5f2edb0 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.135+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01c00f0 con 0x55aae5f28e30 2024-05-27T07:06:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.135+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01a0d20 con 0x55aae5da9ec0 2024-05-27T07:06:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.136+0000 7ff307115b80 10 client.0 _ll_register_callbacks cb 0x55aae5df0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 -- 172.21.15.16:0/1497557244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 msgr2=0x55aae5f47d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f47d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 -- 172.21.15.16:0/1497557244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 msgr2=0x55aae5f455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae5f45b00 con 0x55aae5f28e30 2024-05-27T07:06:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2f5200700 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005cf0 tx=0x7ff2dc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e283085089a14c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.137+0000 7ff2d7400700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:06:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.138+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020040 con 0x55aae5f28e30 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.138+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff2dc0a5930 con 0x55aae5f28e30 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.138+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e4005530 con 0x55aae5f28e30 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.139+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc0ad6f0 con 0x55aae5f28e30 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.139+0000 7ff307115b80 10 client.35960 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.139+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aae5facf60 con 0x55aae5f28e30 2024-05-27T07:06:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.139+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e4005530 con 0x55aae5f28e30 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff2dc0a9630 con 0x55aae5f28e30 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 10 client.35960 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 10 client.35960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 20 client.35960 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5fad610 0x55aae5fafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 4 client.35960 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:06:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.140+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aae5e525f0 con 0x55aae5fad610 2024-05-27T07:06:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.141+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5fad610 0x55aae5fafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.145+0000 7ff2f5c00700 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5fad610 0x55aae5fafa30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec4016e316663d7 server_cookie=18bf3561ff96ef5d in_seq=0 out_seq=0 2024-05-27T07:06:36.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.145+0000 7ff2d7400700 10 client.35960 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.147+0000 7ff2d7400700 1 -- 172.21.15.16:0/1497557244 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aae5e525f0 con 0x55aae5fad610 2024-05-27T07:06:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.147+0000 7ff2d7400700 10 client.35960 handle_command_reply: tid=0 2024-05-27T07:06:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.147+0000 7ff307115b80 1 client.35960 shutdown 2024-05-27T07:06:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.149+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 msgr2=0x55aae5f45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005cf0 tx=0x7ff2dc009f70 comp rx=0 tx=0).stop 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 shutdown_connections 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55aae5fad610 0x55aae5fafa30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aae5f2edb0 0x55aae5f455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aae5da9ec0 0x55aae5f47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 --2- 172.21.15.16:0/1497557244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aae5f28e30 0x55aae5f45080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 >> 172.21.15.16:0/1497557244 conn(0x55aae5e0a040 msgr2=0x55aae5f32e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff2d7400700 0 client.35960 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 shutdown_connections 2024-05-27T07:06:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 1 -- 172.21.15.16:0/1497557244 wait complete. 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.150+0000 7ff307115b80 20 client.35960 trim_cache size 0 max 16384 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5eae6506-9082-437d-876e-21089b1ffcb2", 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:06:36.143 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.151+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 msgr2=0x7ff2d80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.151+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 0x7ff2d80b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009510 tx=0x7ff2e8009040 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.151+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 msgr2=0x7ff2f01a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.151+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d980 tx=0x7ff2dc00dc90 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 shutdown_connections 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff2d80b2a70 0x7ff2d80b4f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2f0109840 0x7ff2f01ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f01045c0 0x7ff2f01a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 --2- 172.21.15.16:0/847736702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f0108ef0 0x7ff2f01a2590 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 >> 172.21.15.16:0/847736702 conn(0x7ff2f00ffe90 msgr2=0x7ff2f010a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.152+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 shutdown_connections 2024-05-27T07:06:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:36.153+0000 7ff2f8600700 1 -- 172.21.15.16:0/847736702 wait complete. 2024-05-27T07:06:37.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:06:37.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:37 smithi016 ceph-mon[31415]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 651 op/s 2024-05-27T07:06:37.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:37 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:06:37.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:37 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:37.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:37 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:06:37.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:37 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:37.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:37 smithi164 ceph-mon[35327]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 651 op/s 2024-05-27T07:06:37.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:37 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:06:37.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:37 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:37.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:37 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:06:37.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:37 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:37.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:37 smithi069 ceph-mon[35276]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 651 op/s 2024-05-27T07:06:37.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:37 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:06:37.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:37 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:37.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:37 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:06:37.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:37 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:06:38.055 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:39 smithi016 ceph-mon[31415]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:06:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:39 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:06:39.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:39 smithi069 ceph-mon[35276]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:06:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:39 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:06:39.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:39 smithi164 ceph-mon[35327]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:06:39.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:39 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:06:41.322 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:41 smithi016 ceph-mon[31415]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 MiB/s wr, 537 op/s 2024-05-27T07:06:41.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:41 smithi069 ceph-mon[35276]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 MiB/s wr, 537 op/s 2024-05-27T07:06:41.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:41 smithi164 ceph-mon[35327]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 MiB/s wr, 537 op/s 2024-05-27T07:06:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.171+0000 7f7a50200700 1 -- 172.21.15.16:0/3753093104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 msgr2=0x7f7a4810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.171+0000 7f7a50200700 1 --2- 172.21.15.16:0/3753093104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a4810bca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a400099c0 tx=0x7f7a40009cd0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.171+0000 7f7a50200700 1 -- 172.21.15.16:0/3753093104 shutdown_connections 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 --2- 172.21.15.16:0/3753093104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 0x7f7a481177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 --2- 172.21.15.16:0/3753093104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a4810bca0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 --2- 172.21.15.16:0/3753093104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 0x7f7a48110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 -- 172.21.15.16:0/3753093104 >> 172.21.15.16:0/3753093104 conn(0x7f7a4807a010 msgr2=0x7f7a4807c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 -- 172.21.15.16:0/3753093104 shutdown_connections 2024-05-27T07:06:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.172+0000 7f7a50200700 1 -- 172.21.15.16:0/3753093104 wait complete. 2024-05-27T07:06:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.178+0000 7f7a50200700 1 Processor -- start 2024-05-27T07:06:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.178+0000 7f7a50200700 1 -- start start 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.180+0000 7f7a50200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.181+0000 7f7a50200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 0x7f7a481a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.181+0000 7f7a50200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 0x7f7a481a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.181+0000 7f7a50200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811a9e0 con 0x7f7a48110fa0 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.181+0000 7f7a50200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811a860 con 0x7f7a48110650 2024-05-27T07:06:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.181+0000 7f7a50200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811ab60 con 0x7f7a4810b890 2024-05-27T07:06:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.182+0000 7f7a4d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.182+0000 7f7a4ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 0x7f7a481a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.182+0000 7f7a4d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48806/0 (socket says 172.21.15.16:48806) 2024-05-27T07:06:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.182+0000 7f7a4d800700 1 -- 172.21.15.16:0/2232035332 learned_addr learned my addr 172.21.15.16:0/2232035332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.183+0000 7f7a4e200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 0x7f7a481a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.184+0000 7f7a4d800700 1 -- 172.21.15.16:0/2232035332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 msgr2=0x7f7a481a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.184+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 0x7f7a481a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.184+0000 7f7a4d800700 1 -- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 msgr2=0x7f7a481a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.184+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 0x7f7a481a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.184+0000 7f7a4d800700 1 -- 172.21.15.16:0/2232035332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481a6300 con 0x7f7a4810b890 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.185+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b580 tx=0x7f7a4000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.186+0000 7f7a46000700 1 -- 172.21.15.16:0/2232035332 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40026070 con 0x7f7a4810b890 2024-05-27T07:06:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.186+0000 7f7a46000700 1 -- 172.21.15.16:0/2232035332 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a4000bd80 con 0x7f7a4810b890 2024-05-27T07:06:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.186+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481a6590 con 0x7f7a4810b890 2024-05-27T07:06:42.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.186+0000 7f7a46000700 1 -- 172.21.15.16:0/2232035332 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40021e10 con 0x7f7a4810b890 2024-05-27T07:06:42.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.186+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481adf10 con 0x7f7a4810b890 2024-05-27T07:06:42.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.193+0000 7f7a46000700 1 -- 172.21.15.16:0/2232035332 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7a400050d0 con 0x7f7a4810b890 2024-05-27T07:06:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.196+0000 7f7a46000700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 0x7f7a2c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.197+0000 7f7a46000700 1 -- 172.21.15.16:0/2232035332 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7a400e43c0 con 0x7f7a4810b890 2024-05-27T07:06:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.198+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 0x7f7a2c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.199+0000 7f7a5eceab80 1 Processor -- start 2024-05-27T07:06:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.199+0000 7f7a5eceab80 1 -- start start 2024-05-27T07:06:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b025330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 0x7f7a2c0b4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a4807bd40 tx=0x7f7a3400b410 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b032110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55680af53e40 con 0x55680af929f0 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55680af439e0 con 0x55680aea6e40 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.200+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55680af47a20 con 0x55680b025e70 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.201+0000 7f7a4ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.201+0000 7f7a4d800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b025330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.201+0000 7f7a4ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:49522/0 (socket says 172.21.15.16:49522) 2024-05-27T07:06:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.201+0000 7f7a4ce00700 1 -- 172.21.15.16:0/486921197 learned_addr learned my addr 172.21.15.16:0/486921197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.201+0000 7f7a4e200700 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b032110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.202+0000 7f7a4ce00700 1 -- 172.21.15.16:0/486921197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 msgr2=0x55680b032110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.202+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b032110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.202+0000 7f7a4ce00700 1 -- 172.21.15.16:0/486921197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 msgr2=0x55680b025330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.202+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b025330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.202+0000 7f7a4ce00700 1 -- 172.21.15.16:0/486921197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680af0c420 con 0x55680aea6e40 2024-05-27T07:06:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.203+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a34010f40 tx=0x7f7a34013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cd180940993b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.204+0000 7f7a3ea00700 1 -- 172.21.15.16:0/486921197 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400be60 con 0x55680aea6e40 2024-05-27T07:06:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.204+0000 7f7a3ea00700 1 -- 172.21.15.16:0/486921197 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a34011db0 con 0x55680aea6e40 2024-05-27T07:06:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.204+0000 7f7a3ea00700 1 -- 172.21.15.16:0/486921197 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3401c650 con 0x55680aea6e40 2024-05-27T07:06:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.206+0000 7f7a5eceab80 1 -- 172.21.15.16:0/486921197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 msgr2=0x55680b025870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.206+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a34010f40 tx=0x7f7a34013000 comp rx=0 tx=0).stop 2024-05-27T07:06:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 -- 172.21.15.16:0/486921197 shutdown_connections 2024-05-27T07:06:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b032110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b025330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/486921197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b025870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 -- 172.21.15.16:0/486921197 >> 172.21.15.16:0/486921197 conn(0x55680af40390 msgr2=0x55680b0214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.208+0000 7f7a5eceab80 1 -- 172.21.15.16:0/486921197 shutdown_connections 2024-05-27T07:06:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.209+0000 7f7a5eceab80 1 -- 172.21.15.16:0/486921197 wait complete. 2024-05-27T07:06:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.209+0000 7f7a5eceab80 1 -- start start 2024-05-27T07:06:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.210+0000 7f7a5eceab80 1 Processor -- start 2024-05-27T07:06:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.212+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b02f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.212+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b02fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55680ae49fa0 con 0x55680af929f0 2024-05-27T07:06:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55680aee4760 con 0x55680aea6e40 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55680b032b80 con 0x55680b025e70 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a4e200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a4e200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48824/0 (socket says 172.21.15.16:48824) 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a4e200700 1 -- 172.21.15.16:0/1196782977 learned_addr learned my addr 172.21.15.16:0/1196782977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.213+0000 7f7a5eceab80 10 client.0 _ll_register_callbacks cb 0x55680aeedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4e200700 1 -- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 msgr2=0x55680b02f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b02fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4d800700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b02f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4e200700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b02f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4e200700 1 -- 172.21.15.16:0/1196782977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 msgr2=0x55680b02fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4e200700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b02fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4e200700 1 -- 172.21.15.16:0/1196782977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680ae938d0 con 0x55680b025e70 2024-05-27T07:06:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.214+0000 7f7a4d800700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b02f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.215+0000 7f7a4e200700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a38008ed0 tx=0x7f7a380114a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfa7ca0782fe0f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.215+0000 7f7a33400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:06:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.215+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38016440 con 0x55680b025e70 2024-05-27T07:06:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.216+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7a380165e0 con 0x55680b025e70 2024-05-27T07:06:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.216+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a34011db0 con 0x55680b025e70 2024-05-27T07:06:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.218+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a38024050 con 0x55680b025e70 2024-05-27T07:06:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.218+0000 7f7a5eceab80 10 client.25912 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.218+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55680af14060 con 0x55680b025e70 2024-05-27T07:06:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a34011db0 con 0x55680b025e70 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7a380155c0 con 0x55680b025e70 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a5eceab80 10 client.25912 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a5eceab80 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a5eceab80 20 client.25912 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.219+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b03fb90 0x55680b042010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.220+0000 7f7a5eceab80 4 client.25912 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:06:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.220+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55680af55d90 con 0x55680b03fb90 2024-05-27T07:06:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.220+0000 7f7a4d800700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b03fb90 0x55680b042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.221+0000 7f7a4d800700 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b03fb90 0x55680b042010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a58f4f903161e6 server_cookie=de0ed94ac97fa4f2 in_seq=0 out_seq=0 2024-05-27T07:06:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.221+0000 7f7a33400700 10 client.25912 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.223+0000 7f7a33400700 1 -- 172.21.15.16:0/1196782977 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55680af55d90 con 0x55680b03fb90 2024-05-27T07:06:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.223+0000 7f7a33400700 10 client.25912 handle_command_reply: tid=0 2024-05-27T07:06:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.224+0000 7f7a5eceab80 1 client.25912 shutdown 2024-05-27T07:06:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.224+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 msgr2=0x55680b0301d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.224+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a38008ed0 tx=0x7f7a380114a0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 shutdown_connections 2024-05-27T07:06:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b03fb90 0x55680b042010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0301d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680aea6e40 0x55680b02f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/1196782977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680af929f0 0x55680b02fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a33400700 0 client.25912 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 >> 172.21.15.16:0/1196782977 conn(0x55680af40390 msgr2=0x55680b0253f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 shutdown_connections 2024-05-27T07:06:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 1 -- 172.21.15.16:0/1196782977 wait complete. 2024-05-27T07:06:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.225+0000 7f7a5eceab80 20 client.25912 trim_cache size 0 max 16384 2024-05-27T07:06:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.276+0000 7f7a5eceab80 1 Processor -- start 2024-05-27T07:06:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.278+0000 7f7a5eceab80 1 -- start start 2024-05-27T07:06:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 0x55680b02af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800ee00 con 0x55680aea6e40 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28010940 con 0x55680af929f0 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.279+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800f4d0 con 0x55680b025e70 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.280+0000 7f7a4ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.280+0000 7f7a4ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:49546/0 (socket says 172.21.15.16:49546) 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.280+0000 7f7a4ce00700 1 -- 172.21.15.16:0/237827059 learned_addr learned my addr 172.21.15.16:0/237827059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.280+0000 7f7a4e200700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.281+0000 7f7a4e200700 1 -- 172.21.15.16:0/237827059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 msgr2=0x55680b03bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.283+0000 7f7a4e200700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.283+0000 7f7a4e200700 1 -- 172.21.15.16:0/237827059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 msgr2=0x55680b02af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.282+0000 7f7a4d800700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 0x55680b02af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.283+0000 7f7a4e200700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 0x55680b02af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.283+0000 7f7a4e200700 1 -- 172.21.15.16:0/237827059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680b01d240 con 0x55680b025e70 2024-05-27T07:06:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.284+0000 7f7a4e200700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0275e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a380159b0 tx=0x7f7a380119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37617b2f6437a656 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.284+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.285+0000 7f7a4d800700 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 0x55680b02af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:06:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.285+0000 7f7a3e000700 1 -- 172.21.15.16:0/237827059 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802c020 con 0x55680b025e70 2024-05-27T07:06:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.285+0000 7f7a3e000700 1 -- 172.21.15.16:0/237827059 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a3801b070 con 0x55680b025e70 2024-05-27T07:06:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.285+0000 7f7a3e000700 1 -- 172.21.15.16:0/237827059 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38004d80 con 0x55680b025e70 2024-05-27T07:06:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.286+0000 7f7a5eceab80 1 -- 172.21.15.16:0/237827059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 msgr2=0x55680b0275e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.286+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0275e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a380159b0 tx=0x7f7a380119d0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 -- 172.21.15.16:0/237827059 shutdown_connections 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680b025e70 0x55680b0275e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680aea6e40 0x55680b02af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/237827059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b03bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 -- 172.21.15.16:0/237827059 >> 172.21.15.16:0/237827059 conn(0x55680af40390 msgr2=0x55680b026ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.287+0000 7f7a5eceab80 1 -- 172.21.15.16:0/237827059 shutdown_connections 2024-05-27T07:06:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.288+0000 7f7a5eceab80 1 -- 172.21.15.16:0/237827059 wait complete. 2024-05-27T07:06:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.288+0000 7f7a5eceab80 1 -- start start 2024-05-27T07:06:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.289+0000 7f7a5eceab80 1 Processor -- start 2024-05-27T07:06:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b0386e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 0x55680b029d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28010270 con 0x55680b025e70 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800fba0 con 0x55680af929f0 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800cda0 con 0x55680aea6e40 2024-05-27T07:06:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.290+0000 7f7a5eceab80 10 client.0 _ll_register_callbacks cb 0x55680aeedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:06:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.292+0000 7f7a4d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.292+0000 7f7a4d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48836/0 (socket says 172.21.15.16:48836) 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.292+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b0386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.292+0000 7f7a4e200700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 0x55680b029d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4e200700 1 -- 172.21.15.16:0/2166296133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 msgr2=0x55680b038180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4e200700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4e200700 1 -- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 msgr2=0x55680b0386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4e200700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b0386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4e200700 1 -- 172.21.15.16:0/2166296133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680b02a530 con 0x55680b025e70 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.293+0000 7f7a4ce00700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b0386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.294+0000 7f7a4d800700 1 -- 172.21.15.16:0/2166296133 learned_addr learned my addr 172.21.15.16:0/2166296133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.294+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-27T07:06:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.294+0000 7f7a4e200700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 0x55680b029d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7a38015930 tx=0x7f7a38015e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8452d50ca2f94e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.294+0000 7f7a33400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:06:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.295+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802c040 con 0x55680b025e70 2024-05-27T07:06:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.295+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7a380155a0 con 0x55680b025e70 2024-05-27T07:06:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.295+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3801b070 con 0x55680b025e70 2024-05-27T07:06:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.296+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a38004a90 con 0x55680b025e70 2024-05-27T07:06:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.296+0000 7f7a5eceab80 10 client.16191 fetch_fsmap learned FSMap version 31 2024-05-27T07:06:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.296+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55680b037950 con 0x55680b025e70 2024-05-27T07:06:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a3801b070 con 0x55680b025e70 2024-05-27T07:06:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7a38004230 con 0x55680b025e70 2024-05-27T07:06:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 10 client.16191 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:06:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:06:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 20 client.16191 populate_metadata read hostname 'smithi016' 2024-05-27T07:06:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b0bb860 0x55680b0bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 4 client.16191 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:06:42.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.297+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55680af4f570 con 0x55680b0bb860 2024-05-27T07:06:42.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.298+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b0bb860 0x55680b0bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:42.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.299+0000 7f7a33400700 10 client.16191 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:42.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.300+0000 7f7a4d800700 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b0bb860 0x55680b0bdc80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e253113e6dd97035 server_cookie=f491bcdcc5c97be3 in_seq=0 out_seq=0 2024-05-27T07:06:42.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.301+0000 7f7a33400700 1 -- 172.21.15.16:0/2166296133 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55680af4f570 con 0x55680b0bb860 2024-05-27T07:06:42.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.301+0000 7f7a33400700 10 client.16191 handle_command_reply: tid=0 2024-05-27T07:06:42.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.301+0000 7f7a5eceab80 1 client.16191 shutdown 2024-05-27T07:06:42.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.301+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 msgr2=0x55680b029d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.301+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 0x55680b029d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7a38015930 tx=0x7f7a38015e80 comp rx=0 tx=0).stop 2024-05-27T07:06:42.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 shutdown_connections 2024-05-27T07:06:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55680b0bb860 0x55680b0bdc80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680b025e70 0x55680b029d60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680aea6e40 0x55680b038180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 --2- 172.21.15.16:0/2166296133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55680af929f0 0x55680b0386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.302+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 >> 172.21.15.16:0/2166296133 conn(0x55680af40390 msgr2=0x55680b02dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.303+0000 7f7a33400700 0 client.16191 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:06:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.304+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 shutdown_connections 2024-05-27T07:06:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.304+0000 7f7a5eceab80 1 -- 172.21.15.16:0/2166296133 wait complete. 2024-05-27T07:06:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.304+0000 7f7a5eceab80 20 client.16191 trim_cache size 0 max 16384 2024-05-27T07:06:42.297 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:06:42.297 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:06:42.297 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:06:42.298 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:06:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 msgr2=0x7f7a2c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 0x7f7a2c0b4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a4807bd40 tx=0x7f7a3400b410 comp rx=0 tx=0).stop 2024-05-27T07:06:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 msgr2=0x7f7a481a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b580 tx=0x7f7a4000f690 comp rx=0 tx=0).stop 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 shutdown_connections 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f7a2c0b2aa0 0x7f7a2c0b4fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a48110fa0 0x7f7a481a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a4810b890 0x7f7a481a1430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 --2- 172.21.15.16:0/2232035332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a48110650 0x7f7a481a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.306+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 >> 172.21.15.16:0/2232035332 conn(0x7f7a4807a010 msgr2=0x7f7a4807c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.307+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 shutdown_connections 2024-05-27T07:06:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:42.307+0000 7f7a50200700 1 -- 172.21.15.16:0/2232035332 wait complete. 2024-05-27T07:06:43.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:06:43.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:06:43.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:43 smithi016 ceph-mon[31415]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-05-27T07:06:43.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:43 smithi069 ceph-mon[35276]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-05-27T07:06:43.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:43 smithi164 ceph-mon[35327]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-05-27T07:06:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:45.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:45 smithi164 ceph-mon[35327]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:45.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:45 smithi069 ceph-mon[35276]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:45.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:45 smithi016 ceph-mon[31415]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 29 MiB/s wr, 464 op/s 2024-05-27T07:06:47.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:47 smithi164 ceph-mon[35327]: pgmap v179: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:47.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:47 smithi016 ceph-mon[31415]: pgmap v179: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:47.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:47 smithi069 ceph-mon[35276]: pgmap v179: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 41 MiB/s wr, 667 op/s 2024-05-27T07:06:49.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:49 smithi164 ceph-mon[35327]: pgmap v180: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:06:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:49 smithi016 ceph-mon[31415]: pgmap v180: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:06:49.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:49 smithi069 ceph-mon[35276]: pgmap v180: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 31 MiB/s wr, 510 op/s 2024-05-27T07:06:51.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:51 smithi164 ceph-mon[35327]: pgmap v181: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:06:51.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:51 smithi016 ceph-mon[31415]: pgmap v181: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:06:51.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:51 smithi069 ceph-mon[35276]: pgmap v181: 129 pgs: 129 active+clean; 14 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:06:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.960+0000 7f75ce800700 1 -- 172.21.15.16:0/810900590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8108730 msgr2=0x7f75c8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.960+0000 7f75ce800700 1 --2- 172.21.15.16:0/810900590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8108730 0x7f75c8110090 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009fc0 tx=0x7f75c000b330 comp rx=0 tx=0).stop 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 -- 172.21.15.16:0/810900590 shutdown_connections 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 --2- 172.21.15.16:0/810900590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8108730 0x7f75c8110090 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 --2- 172.21.15.16:0/810900590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c81034b0 0x7f75c81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 --2- 172.21.15.16:0/810900590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c8107de0 0x7f75c81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 -- 172.21.15.16:0/810900590 >> 172.21.15.16:0/810900590 conn(0x7f75c80fedb0 msgr2=0x7f75c81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 -- 172.21.15.16:0/810900590 shutdown_connections 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.962+0000 7f75ce800700 1 -- 172.21.15.16:0/810900590 wait complete. 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.965+0000 7f75ce800700 1 Processor -- start 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.965+0000 7f75ce800700 1 -- start start 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c81034b0 0x7f75c80753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 0x7f75c806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81121a0 con 0x7f75c8107de0 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8112020 con 0x7f75c81034b0 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75ce800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8112320 con 0x7f75c8108730 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.966+0000 7f75c7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c81034b0 0x7f75c80753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.967+0000 7f75c6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.967+0000 7f75c6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52696/0 (socket says 172.21.15.16:52696) 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.967+0000 7f75c6a00700 1 -- 172.21.15.16:0/2284668974 learned_addr learned my addr 172.21.15.16:0/2284668974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.967+0000 7f75c7e00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 0x7f75c806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 -- 172.21.15.16:0/2284668974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c81034b0 msgr2=0x7f75c80753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c81034b0 0x7f75c80753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 -- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 msgr2=0x7f75c806e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 -- 172.21.15.16:0/2284668974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c806f320 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c6a00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.968+0000 7f75c7e00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 0x7f75c806eb50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75c0015000 tx=0x7f75c0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.970+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0021070 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.970+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c0007ac0 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.970+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00052a0 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.970+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c8072fb0 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.971+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c80734a0 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.974+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c8104960 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.979+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f75c0024070 con 0x7f75c8108730 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.981+0000 7f75bfe00700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 0x7f75ac0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:06:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.982+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f75c00a6e10 con 0x7f75c8108730 2024-05-27T07:06:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.983+0000 7f75c7400700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 0x7f75ac0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:06:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.984+0000 7f75c7400700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 0x7f75ac0b4fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75c806fd40 tx=0x7f75b8005d70 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:06:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:52.993+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75c002b070 con 0x7f75c8108730 2024-05-27T07:06:53.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:53 smithi016 ceph-mon[31415]: pgmap v182: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 39 MiB/s wr, 634 op/s 2024-05-27T07:06:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.575+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75c806fa20 con 0x7f75c8108730 2024-05-27T07:06:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.576+0000 7f75bfe00700 1 -- 172.21.15.16:0/2284668974 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f75c00aaf20 con 0x7f75c8108730 2024-05-27T07:06:53.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:06:53.571 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.581+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 msgr2=0x7f75ac0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.581+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 0x7f75ac0b4fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75c806fd40 tx=0x7f75b8005d70 comp rx=0 tx=0).stop 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.581+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 msgr2=0x7f75c806eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.581+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 0x7f75c806eb50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75c0015000 tx=0x7f75c0009510 comp rx=0 tx=0).stop 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 shutdown_connections 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f75ac0b2aa0 0x7f75ac0b4fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75c8108730 0x7f75c806eb50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75c81034b0 0x7f75c80753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 --2- 172.21.15.16:0/2284668974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8107de0 0x7f75c806e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:06:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.582+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 >> 172.21.15.16:0/2284668974 conn(0x7f75c80fedb0 msgr2=0x7f75c8109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:06:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.585+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 shutdown_connections 2024-05-27T07:06:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:06:53.585+0000 7f75ce800700 1 -- 172.21.15.16:0/2284668974 wait complete. 2024-05-27T07:06:53.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:06:53.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:53 smithi164 ceph-mon[35327]: pgmap v182: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 39 MiB/s wr, 634 op/s 2024-05-27T07:06:53.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:53 smithi069 ceph-mon[35276]: pgmap v182: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 39 MiB/s wr, 634 op/s 2024-05-27T07:06:54.580 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:54 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2284668974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:06:54.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:54 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2284668974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:06:54.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:54 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2284668974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:06:54.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:06:55.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:55 smithi016 ceph-mon[31415]: pgmap v183: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 476 op/s 2024-05-27T07:06:55.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:55 smithi164 ceph-mon[35327]: pgmap v183: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 476 op/s 2024-05-27T07:06:55.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:55 smithi069 ceph-mon[35276]: pgmap v183: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 29 MiB/s wr, 476 op/s 2024-05-27T07:06:57.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:57 smithi164 ceph-mon[35327]: pgmap v184: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 684 op/s 2024-05-27T07:06:57.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:57 smithi016 ceph-mon[31415]: pgmap v184: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 684 op/s 2024-05-27T07:06:57.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:57 smithi069 ceph-mon[35276]: pgmap v184: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 684 op/s 2024-05-27T07:06:57.634 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:06:59.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:06:59 smithi016 ceph-mon[31415]: pgmap v185: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:06:59.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:06:59 smithi164 ceph-mon[35327]: pgmap v185: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:06:59.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:06:59 smithi069 ceph-mon[35276]: pgmap v185: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:07:01.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:01 smithi164 ceph-mon[35327]: pgmap v186: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:07:01.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:01 smithi016 ceph-mon[31415]: pgmap v186: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:07:01.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:01 smithi069 ceph-mon[35276]: pgmap v186: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 35 MiB/s wr, 567 op/s 2024-05-27T07:07:03.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:03 smithi016 ceph-mon[31415]: pgmap v187: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 601 op/s 2024-05-27T07:07:03.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:03 smithi164 ceph-mon[35327]: pgmap v187: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 601 op/s 2024-05-27T07:07:03.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:03 smithi069 ceph-mon[35276]: pgmap v187: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 601 op/s 2024-05-27T07:07:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.497+0000 7f2b06e00700 1 -- 172.21.15.16:0/348109623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00103b90 msgr2=0x7f2b001080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.497+0000 7f2b06e00700 1 --2- 172.21.15.16:0/348109623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00103b90 0x7f2b001080c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009990 tx=0x7f2ae8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.796 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 -- 172.21.15.16:0/348109623 shutdown_connections 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 --2- 172.21.15.16:0/348109623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108690 0x7f2b00108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 --2- 172.21.15.16:0/348109623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103240 0x7f2b00103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 --2- 172.21.15.16:0/348109623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00103b90 0x7f2b001080c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 -- 172.21.15.16:0/348109623 >> 172.21.15.16:0/348109623 conn(0x7f2b000fedb0 msgr2=0x7f2b001011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 -- 172.21.15.16:0/348109623 shutdown_connections 2024-05-27T07:07:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.498+0000 7f2b06e00700 1 -- 172.21.15.16:0/348109623 wait complete. 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.500+0000 7f2b06e00700 1 Processor -- start 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.500+0000 7f2b06e00700 1 -- start start 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.500+0000 7f2b06e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 0x7f2b001a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.500+0000 7f2b06e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103b90 0x7f2b001a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.501+0000 7f2b06e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.501+0000 7f2b06e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0011a910 con 0x7f2b00103240 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.501+0000 7f2b06e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0011a790 con 0x7f2b00108690 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.501+0000 7f2b06e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0011aa90 con 0x7f2b00103b90 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.501+0000 7f2b04e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53594/0 (socket says 172.21.15.16:53594) 2024-05-27T07:07:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 -- 172.21.15.16:0/98051205 learned_addr learned my addr 172.21.15.16:0/98051205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2aff400700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103b90 0x7f2b001a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2affe00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 0x7f2b001a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 -- 172.21.15.16:0/98051205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103b90 msgr2=0x7f2b001a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103b90 0x7f2b001a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 -- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 msgr2=0x7f2b001a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 0x7f2b001a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2b04e00700 1 -- 172.21.15.16:0/98051205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b001a6210 con 0x7f2b00108690 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.502+0000 7f2affe00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 0x7f2b001a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.503+0000 7f2b04e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2af000beb0 tx=0x7f2af000dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.503+0000 7f2afcc00700 1 -- 172.21.15.16:0/98051205 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0039070 con 0x7f2b00108690 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.504+0000 7f2afcc00700 1 -- 172.21.15.16:0/98051205 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2af0005020 con 0x7f2b00108690 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.504+0000 7f2afcc00700 1 -- 172.21.15.16:0/98051205 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0039070 con 0x7f2b00108690 2024-05-27T07:07:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.504+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b001a6500 con 0x7f2b00108690 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.504+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001a6930 con 0x7f2b00108690 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.510+0000 7f2afcc00700 1 -- 172.21.15.16:0/98051205 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2af0031770 con 0x7f2b00108690 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.512+0000 7f2afcc00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 0x7f2ae40b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.513+0000 7f2afcc00700 1 -- 172.21.15.16:0/98051205 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2af01051a0 con 0x7f2b00108690 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.513+0000 7f2affe00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 0x7f2ae40b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.517+0000 7f2b1596db80 1 Processor -- start 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.517+0000 7f2b1596db80 1 -- start start 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.518+0000 7f2affe00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 0x7f2ae40b5180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009fd0 tx=0x7f2af800b040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.518+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c7fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.518+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.518+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c818530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2aff400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2aff400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53610/0 (socket says 172.21.15.16:53610) 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2b04e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c818530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2aff400700 1 -- 172.21.15.16:0/1871420548 learned_addr learned my addr 172.21.15.16:0/1871420548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c728e40 con 0x55ed8c816110 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c7189e0 con 0x55ed8c67be40 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.519+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c71ca20 con 0x55ed8c7679f0 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2aff400700 1 -- 172.21.15.16:0/1871420548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 msgr2=0x55ed8c7fa3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2affe00700 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c7fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2aff400700 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c7fa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2aff400700 1 -- 172.21.15.16:0/1871420548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 msgr2=0x55ed8c818530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2aff400700 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c818530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2aff400700 1 -- 172.21.15.16:0/1871420548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c6e1420 con 0x55ed8c67be40 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.520+0000 7f2b04e00700 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c818530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.521+0000 7f2aff400700 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009990 tx=0x7f2ae8005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e88a88521e5dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.521+0000 7f2af5600700 1 -- 172.21.15.16:0/1871420548 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8027070 con 0x55ed8c67be40 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.521+0000 7f2af5600700 1 -- 172.21.15.16:0/1871420548 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ae8003b10 con 0x55ed8c67be40 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.521+0000 7f2af5600700 1 -- 172.21.15.16:0/1871420548 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800bc10 con 0x55ed8c67be40 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 msgr2=0x55ed8c7fa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009990 tx=0x7f2ae8005f00 comp rx=0 tx=0).stop 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 shutdown_connections 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c818530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c7fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1871420548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c7fa910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.523+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 >> 172.21.15.16:0/1871420548 conn(0x55ed8c715390 msgr2=0x55ed8c7f64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.526+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 shutdown_connections 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.526+0000 7f2b1596db80 1 -- 172.21.15.16:0/1871420548 wait complete. 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.527+0000 7f2b1596db80 1 -- start start 2024-05-27T07:07:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.527+0000 7f2b1596db80 1 Processor -- start 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c67be40 0x55ed8c813160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c813be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c61efa0 con 0x55ed8c816110 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c6b9760 con 0x55ed8c7679f0 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.529+0000 7f2b1596db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ed8c818b60 con 0x55ed8c67be40 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.530+0000 7f2b1596db80 10 client.0 _ll_register_callbacks cb 0x55ed8c6c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.530+0000 7f2aff400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.530+0000 7f2aff400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53620/0 (socket says 172.21.15.16:53620) 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.530+0000 7f2aff400700 1 -- 172.21.15.16:0/2863664608 learned_addr learned my addr 172.21.15.16:0/2863664608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 -- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c67be40 msgr2=0x55ed8c813160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c67be40 0x55ed8c813160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 -- 172.21.15.16:0/2863664608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 msgr2=0x55ed8c813be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:07:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c813be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 -- 172.21.15.16:0/2863664608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c6688d0 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aff400700 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6feb5f45cbcc1748 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aee000700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8031060 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.531+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2ae8003880 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.532+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae8003b10 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.532+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ae8027040 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.532+0000 7f2b1596db80 10 client.35990 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.532+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed8c6e9060 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.532+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ae8003b10 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae800b6b0 con 0x55ed8c7679f0 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 10 client.35990 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 10 client.35990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 20 client.35990 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c825a40 0x55ed8c827f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 4 client.35990 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.534+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8c72ad90 con 0x55ed8c825a40 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.535+0000 7f2affe00700 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c825a40 0x55ed8c827f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.536+0000 7f2aee000700 10 client.35990 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.537+0000 7f2affe00700 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c825a40 0x55ed8c827f50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b71513e0b54e5c8 server_cookie=2146f43a6d7cec32 in_seq=0 out_seq=0 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.539+0000 7f2aee000700 1 -- 172.21.15.16:0/2863664608 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed8c72ad90 con 0x55ed8c825a40 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.540+0000 7f2aee000700 10 client.35990 handle_command_reply: tid=0 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.540+0000 7f2b1596db80 1 client.35990 shutdown 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.540+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 msgr2=0x55ed8c8136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.541+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8003cc0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.541+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 shutdown_connections 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.541+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c825a40 0x55ed8c827f50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.541+0000 7f2aee000700 0 client.35990 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.542+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c813be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.542+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c67be40 0x55ed8c813160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.542+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2863664608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c8136a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.542+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 >> 172.21.15.16:0/2863664608 conn(0x55ed8c715390 msgr2=0x55ed8c7fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.543+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 shutdown_connections 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.543+0000 7f2b1596db80 1 -- 172.21.15.16:0/2863664608 wait complete. 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.543+0000 7f2b1596db80 20 client.35990 trim_cache size 0 max 16384 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.573+0000 7f2b1596db80 1 Processor -- start 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.573+0000 7f2b1596db80 1 -- start start 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.573+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c67be40 0x55ed8c80e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c82d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b1596db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00ee30 con 0x55ed8c67be40 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b1596db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc010970 con 0x55ed8c7679f0 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b1596db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00f500 con 0x55ed8c816110 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b04e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b04e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52222/0 (socket says 172.21.15.16:52222) 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.574+0000 7f2b04e00700 1 -- 172.21.15.16:0/1067549064 learned_addr learned my addr 172.21.15.16:0/1067549064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.575+0000 7f2aff400700 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c82d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.575+0000 7f2affe00700 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c67be40 0x55ed8c80e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.578+0000 7f2b04e00700 1 -- 172.21.15.16:0/1067549064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 msgr2=0x55ed8c82d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.578+0000 7f2b04e00700 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c82d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.578+0000 7f2b04e00700 1 -- 172.21.15.16:0/1067549064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c67be40 msgr2=0x55ed8c80e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.578+0000 7f2b04e00700 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c67be40 0x55ed8c80e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.578+0000 7f2b04e00700 1 -- 172.21.15.16:0/1067549064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8c814170 con 0x55ed8c816110 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.581+0000 7f2b04e00700 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2af00cbdf0 tx=0x7f2af00cc270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac2e14e75a0a94b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.581+0000 7f2af4c00700 1 -- 172.21.15.16:0/1067549064 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af00e6070 con 0x55ed8c816110 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.581+0000 7f2af4c00700 1 -- 172.21.15.16:0/1067549064 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2af00cc560 con 0x55ed8c816110 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.581+0000 7f2af4c00700 1 -- 172.21.15.16:0/1067549064 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af00cb5b0 con 0x55ed8c816110 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 -- 172.21.15.16:0/1067549064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 msgr2=0x55ed8c82dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2af00cbdf0 tx=0x7f2af00cc270 comp rx=0 tx=0).stop 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 -- 172.21.15.16:0/1067549064 shutdown_connections 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c816110 0x55ed8c82dc90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c67be40 0x55ed8c80e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 --2- 172.21.15.16:0/1067549064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c7679f0 0x55ed8c82d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.583+0000 7f2b1596db80 1 -- 172.21.15.16:0/1067549064 >> 172.21.15.16:0/1067549064 conn(0x55ed8c715390 msgr2=0x55ed8c80e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.584+0000 7f2b1596db80 1 -- 172.21.15.16:0/1067549064 shutdown_connections 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.584+0000 7f2b1596db80 1 -- 172.21.15.16:0/1067549064 wait complete. 2024-05-27T07:07:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.585+0000 7f2b1596db80 1 -- start start 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.585+0000 7f2b1596db80 1 Processor -- start 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c81e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2affe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c81e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2affe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c81e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53642/0 (socket says 172.21.15.16:53642) 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2aff400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2aff400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:52232/0 (socket says 172.21.15.16:52232) 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.586+0000 7f2b1596db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c816c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2b1596db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0102a0 con 0x55ed8c816110 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2b1596db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00fbd0 con 0x55ed8c67be40 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2b1596db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00cd90 con 0x55ed8c7679f0 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2b1596db80 10 client.0 _ll_register_callbacks cb 0x55ed8c6c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2affe00700 1 -- 172.21.15.16:0/2043363984 learned_addr learned my addr 172.21.15.16:0/2043363984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.587+0000 7f2b04e00700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c816c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 -- 172.21.15.16:0/2043363984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 msgr2=0x55ed8c81e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c81e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 -- 172.21.15.16:0/2043363984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 msgr2=0x55ed8c816c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c816c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 -- 172.21.15.16:0/2043363984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8c813f50 con 0x55ed8c7679f0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2b04e00700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c816c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aff400700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8021b80 tx=0x7f2ae8022690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c200c4fe7b5790 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.588+0000 7f2aee000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae803a020 con 0x55ed8c7679f0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2ae800bd50 con 0x55ed8c7679f0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af00cc560 con 0x55ed8c7679f0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ae8027040 con 0x55ed8c7679f0 2024-05-27T07:07:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2b1596db80 10 client.25930 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.589+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed8c81d770 con 0x55ed8c7679f0 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.590+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2af00cc560 con 0x55ed8c7679f0 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae800b6b0 con 0x55ed8c7679f0 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 10 client.25930 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 10 client.25930 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 20 client.25930 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c890620 0x55ed8c892a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 4 client.25930 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed8c724570 con 0x55ed8c890620 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.591+0000 7f2affe00700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c890620 0x55ed8c892a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.592+0000 7f2affe00700 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c890620 0x55ed8c892a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4b43c8817167192 server_cookie=308ea18f05e79571 in_seq=0 out_seq=0 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.592+0000 7f2aee000700 10 client.25930 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:07:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.593+0000 7f2aee000700 1 -- 172.21.15.16:0/2043363984 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed8c724570 con 0x55ed8c890620 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.593+0000 7f2aee000700 10 client.25930 handle_command_reply: tid=0 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.593+0000 7f2b1596db80 1 client.25930 shutdown 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.593+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 msgr2=0x55ed8c81e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.593+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8021b80 tx=0x7f2ae8022690 comp rx=0 tx=0).stop 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.594+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 shutdown_connections 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55ed8c890620 0x55ed8c892a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed8c816110 0x55ed8c816c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed8c67be40 0x55ed8c81e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2aee000700 0 client.25930 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2b1596db80 1 --2- 172.21.15.16:0/2043363984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed8c7679f0 0x55ed8c81e650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.595+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 >> 172.21.15.16:0/2043363984 conn(0x55ed8c715390 msgr2=0x55ed8c811a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.596+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 shutdown_connections 2024-05-27T07:07:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.596+0000 7f2b1596db80 1 -- 172.21.15.16:0/2043363984 wait complete. 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.597+0000 7f2b1596db80 20 client.25930 trim_cache size 0 max 16384 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.598+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 msgr2=0x7f2ae40b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.598+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 0x7f2ae40b5180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009fd0 tx=0x7f2af800b040 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.598+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 msgr2=0x7f2b001a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.598+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2af000beb0 tx=0x7f2af000dbb0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 shutdown_connections 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2ae40b2c70 0x7f2ae40b5180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b00108690 0x7f2b001a5b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00103240 0x7f2b001a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 --2- 172.21.15.16:0/98051205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00103b90 0x7f2b001a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 >> 172.21.15.16:0/98051205 conn(0x7f2b000fedb0 msgr2=0x7f2b00105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 shutdown_connections 2024-05-27T07:07:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:04.599+0000 7f2b06e00700 1 -- 172.21.15.16:0/98051205 wait complete. 2024-05-27T07:07:05.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:05 smithi016 ceph-mon[31415]: pgmap v188: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 490 op/s 2024-05-27T07:07:05.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:05 smithi164 ceph-mon[35327]: pgmap v188: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 490 op/s 2024-05-27T07:07:05.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:05 smithi069 ceph-mon[35276]: pgmap v188: 129 pgs: 129 active+clean; 14 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 490 op/s 2024-05-27T07:07:07.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:07 smithi164 ceph-mon[35327]: pgmap v189: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 686 op/s 2024-05-27T07:07:07.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:07 smithi016 ceph-mon[31415]: pgmap v189: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 686 op/s 2024-05-27T07:07:07.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:07 smithi069 ceph-mon[35276]: pgmap v189: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 686 op/s 2024-05-27T07:07:08.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:07:09.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:09 smithi164 ceph-mon[35327]: pgmap v190: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 507 op/s 2024-05-27T07:07:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:09 smithi016 ceph-mon[31415]: pgmap v190: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 507 op/s 2024-05-27T07:07:09.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:09 smithi069 ceph-mon[35276]: pgmap v190: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 507 op/s 2024-05-27T07:07:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:07:11.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:11 smithi164 ceph-mon[35327]: pgmap v191: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 498 op/s 2024-05-27T07:07:11.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:11 smithi016 ceph-mon[31415]: pgmap v191: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 498 op/s 2024-05-27T07:07:11.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:11 smithi069 ceph-mon[35276]: pgmap v191: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 MiB/s wr, 498 op/s 2024-05-27T07:07:13.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:13 smithi164 ceph-mon[35327]: pgmap v192: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 594 op/s 2024-05-27T07:07:13.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:13 smithi016 ceph-mon[31415]: pgmap v192: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 594 op/s 2024-05-27T07:07:13.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:13 smithi069 ceph-mon[35276]: pgmap v192: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 594 op/s 2024-05-27T07:07:15.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:15 smithi164 ceph-mon[35327]: pgmap v193: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 505 op/s 2024-05-27T07:07:15.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:15 smithi016 ceph-mon[31415]: pgmap v193: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 505 op/s 2024-05-27T07:07:15.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:15 smithi069 ceph-mon[35276]: pgmap v193: 129 pgs: 129 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 505 op/s 2024-05-27T07:07:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.309+0000 7f32ea800700 1 -- 172.21.15.16:0/614854559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 msgr2=0x7f32e41049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.309+0000 7f32ea800700 1 --2- 172.21.15.16:0/614854559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e41049d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b0a0 tx=0x7f32d400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.312+0000 7f32ea800700 1 -- 172.21.15.16:0/614854559 shutdown_connections 2024-05-27T07:07:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.312+0000 7f32ea800700 1 --2- 172.21.15.16:0/614854559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e41111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.312+0000 7f32ea800700 1 --2- 172.21.15.16:0/614854559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e41049d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.312+0000 7f32ea800700 1 --2- 172.21.15.16:0/614854559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 0x7f32e4109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.313+0000 7f32ea800700 1 -- 172.21.15.16:0/614854559 >> 172.21.15.16:0/614854559 conn(0x7f32e40ffe90 msgr2=0x7f32e41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.313+0000 7f32ea800700 1 -- 172.21.15.16:0/614854559 shutdown_connections 2024-05-27T07:07:17.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.315+0000 7f32ea800700 1 -- 172.21.15.16:0/614854559 wait complete. 2024-05-27T07:07:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.318+0000 7f32ea800700 1 Processor -- start 2024-05-27T07:07:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.318+0000 7f32ea800700 1 -- start start 2024-05-27T07:07:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 0x7f32e419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41143e0 con 0x7f32e4108ef0 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4114260 con 0x7f32e41045c0 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32ea800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4114560 con 0x7f32e4109840 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.319+0000 7f32daa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 0x7f32e419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.320+0000 7f32e3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.320+0000 7f32e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.320+0000 7f32e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51730/0 (socket says 172.21.15.16:51730) 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.320+0000 7f32e3400700 1 -- 172.21.15.16:0/3265590418 learned_addr learned my addr 172.21.15.16:0/3265590418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 -- 172.21.15.16:0/3265590418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 msgr2=0x7f32e419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 -- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 msgr2=0x7f32e419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 0x7f32e419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 -- 172.21.15.16:0/3265590418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e419fc80 con 0x7f32e41045c0 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3e00700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:07:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.321+0000 7f32e3400700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f32d4000f80 tx=0x7f32d4004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.327+0000 7f32e0c00700 1 -- 172.21.15.16:0/3265590418 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4021070 con 0x7f32e41045c0 2024-05-27T07:07:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.327+0000 7f32e0c00700 1 -- 172.21.15.16:0/3265590418 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32d4007c00 con 0x7f32e41045c0 2024-05-27T07:07:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.327+0000 7f32e0c00700 1 -- 172.21.15.16:0/3265590418 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40046e0 con 0x7f32e41045c0 2024-05-27T07:07:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.327+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e419ff10 con 0x7f32e41045c0 2024-05-27T07:07:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.327+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41a0400 con 0x7f32e41045c0 2024-05-27T07:07:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.336+0000 7f32e0c00700 1 -- 172.21.15.16:0/3265590418 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f32d401d450 con 0x7f32e41045c0 2024-05-27T07:07:17.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.345+0000 7f32e0c00700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 0x7f32c80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.345+0000 7f32e0c00700 1 -- 172.21.15.16:0/3265590418 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32d400f020 con 0x7f32e41045c0 2024-05-27T07:07:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.347+0000 7f32f93f8b80 1 Processor -- start 2024-05-27T07:07:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.348+0000 7f32daa00700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 0x7f32c80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.348+0000 7f32f93f8b80 1 -- start start 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e94330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e94870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3ea1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3dc2e40 con 0x55e4a3e94e70 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3db29e0 con 0x55e4a3d15e40 2024-05-27T07:07:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.349+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3db6a20 con 0x55e4a3e019f0 2024-05-27T07:07:17.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.351+0000 7f32daa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e94870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32daa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e94870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51738/0 (socket says 172.21.15.16:51738) 2024-05-27T07:07:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32e3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e94330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32daa00700 1 -- 172.21.15.16:0/2232578106 learned_addr learned my addr 172.21.15.16:0/2232578106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32e3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3ea1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32daa00700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 0x7f32c80b50d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32e41075c0 tx=0x7f32cc008040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32e3400700 1 -- 172.21.15.16:0/2232578106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 msgr2=0x55e4a3e94870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32e3400700 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e94870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.352+0000 7f32e3400700 1 -- 172.21.15.16:0/2232578106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 msgr2=0x55e4a3ea1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.353+0000 7f32e3400700 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3ea1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.353+0000 7f32e3400700 1 -- 172.21.15.16:0/2232578106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a3d7b420 con 0x55e4a3e019f0 2024-05-27T07:07:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.353+0000 7f32e3e00700 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3ea1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.353+0000 7f32e3400700 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e94330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32d40e20f0 tx=0x7f32d40ab570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c278f38a1b995bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.354+0000 7f32d8c00700 1 -- 172.21.15.16:0/2232578106 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40e2580 con 0x55e4a3e019f0 2024-05-27T07:07:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.354+0000 7f32d8c00700 1 -- 172.21.15.16:0/2232578106 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32d40b4430 con 0x55e4a3e019f0 2024-05-27T07:07:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.354+0000 7f32d8c00700 1 -- 172.21.15.16:0/2232578106 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40ac1d0 con 0x55e4a3e019f0 2024-05-27T07:07:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2232578106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 msgr2=0x55e4a3e94330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e94330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32d40e20f0 tx=0x7f32d40ab570 comp rx=0 tx=0).stop 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2232578106 shutdown_connections 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3ea1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e94330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2232578106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e94870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.356+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2232578106 >> 172.21.15.16:0/2232578106 conn(0x55e4a3daf390 msgr2=0x55e4a3e904c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.357+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2232578106 shutdown_connections 2024-05-27T07:07:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.357+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2232578106 wait complete. 2024-05-27T07:07:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.358+0000 7f32f93f8b80 1 -- start start 2024-05-27T07:07:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.359+0000 7f32f93f8b80 1 Processor -- start 2024-05-27T07:07:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.360+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e9e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.360+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.360+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3e9f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.361+0000 7f32e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e9e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.361+0000 7f32daa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.361+0000 7f32daa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33078/0 (socket says 172.21.15.16:33078) 2024-05-27T07:07:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.361+0000 7f32e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e9e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51750/0 (socket says 172.21.15.16:51750) 2024-05-27T07:07:17.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.362+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3cb8fa0 con 0x55e4a3e94e70 2024-05-27T07:07:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.362+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3d53760 con 0x55e4a3d15e40 2024-05-27T07:07:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.362+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3eb2950 con 0x55e4a3e019f0 2024-05-27T07:07:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.362+0000 7f32daa00700 1 -- 172.21.15.16:0/1713796111 learned_addr learned my addr 172.21.15.16:0/1713796111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.362+0000 7f32f93f8b80 10 client.0 _ll_register_callbacks cb 0x55e4a3d5cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.364+0000 7f32e3e00700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3e9f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.366+0000 7f32e3e00700 1 -- 172.21.15.16:0/1713796111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 msgr2=0x55e4a3e9ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.366+0000 7f32e3e00700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.366+0000 7f32e3e00700 1 -- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 msgr2=0x55e4a3e9e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.366+0000 7f32e3e00700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e9e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.366+0000 7f32e3e00700 1 -- 172.21.15.16:0/1713796111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a3d028d0 con 0x55e4a3e94e70 2024-05-27T07:07:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.367+0000 7f32d1600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:07:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.367+0000 7f32e3e00700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3e9f200 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f32dc012700 tx=0x7f32dc012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a205eb7c5d39393a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.368+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc01f060 con 0x55e4a3e94e70 2024-05-27T07:07:17.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.369+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d40b4430 con 0x55e4a3e94e70 2024-05-27T07:07:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.369+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x55e4a3e94e70 2024-05-27T07:07:17.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.370+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32dc01f060 con 0x55e4a3e94e70 2024-05-27T07:07:17.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.371+0000 7f32f93f8b80 10 client.16209 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:17.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.371+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4a3d83060 con 0x55e4a3e94e70 2024-05-27T07:07:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.372+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32d40b4430 con 0x55e4a3e94e70 2024-05-27T07:07:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f32dc01a8e0 con 0x55e4a3e94e70 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 10 client.16209 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 20 client.16209 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3ebfaa0 0x55e4a3ec1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 4 client.16209 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a3dc4d90 con 0x55e4a3ebfaa0 2024-05-27T07:07:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.374+0000 7f32e3400700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3ebfaa0 0x55e4a3ec1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.375+0000 7f32d1600700 10 client.16209 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:07:17.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.375+0000 7f32e3400700 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3ebfaa0 0x55e4a3ec1f30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6181ee162c65c09f server_cookie=7ce66f28857fa391 in_seq=0 out_seq=0 2024-05-27T07:07:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.377+0000 7f32d1600700 1 -- 172.21.15.16:0/1713796111 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4a3dc4d90 con 0x55e4a3ebfaa0 2024-05-27T07:07:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.377+0000 7f32d1600700 10 client.16209 handle_command_reply: tid=0 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.377+0000 7f32f93f8b80 1 client.16209 shutdown 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 msgr2=0x55e4a3e9f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3e9f200 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f32dc012700 tx=0x7f32dc012a10 comp rx=0 tx=0).stop 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 shutdown_connections 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e94e70 0x55e4a3e9f200 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3ebfaa0 0x55e4a3ec1f30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3d15e40 0x55e4a3e9e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/1713796111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32d1600700 0 client.16209 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.379+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 >> 172.21.15.16:0/1713796111 conn(0x55e4a3daf390 msgr2=0x55e4a3e94370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.380+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 shutdown_connections 2024-05-27T07:07:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.380+0000 7f32f93f8b80 1 -- 172.21.15.16:0/1713796111 wait complete. 2024-05-27T07:07:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.380+0000 7f32f93f8b80 20 client.16209 trim_cache size 0 max 16384 2024-05-27T07:07:17.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.420+0000 7f32f93f8b80 1 Processor -- start 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.420+0000 7f32f93f8b80 1 -- start start 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3d15e40 0x55e4a3e948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40739c0 con 0x55e4a3e019f0 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4075500 con 0x55e4a3e94e70 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.421+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4074090 con 0x55e4a3d15e40 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.422+0000 7f32e3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.423+0000 7f32e3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51752/0 (socket says 172.21.15.16:51752) 2024-05-27T07:07:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.423+0000 7f32e3e00700 1 -- 172.21.15.16:0/2351747992 learned_addr learned my addr 172.21.15.16:0/2351747992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3400700 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3d15e40 0x55e4a3e948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32daa00700 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3e00700 1 -- 172.21.15.16:0/2351747992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3d15e40 msgr2=0x55e4a3e948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3e00700 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3d15e40 0x55e4a3e948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3e00700 1 -- 172.21.15.16:0/2351747992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e019f0 msgr2=0x55e4a3e9d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3e00700 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.424+0000 7f32e3e00700 1 -- 172.21.15.16:0/2351747992 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a3e9f570 con 0x55e4a3e94e70 2024-05-27T07:07:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.425+0000 7f32e3e00700 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32dc001eb0 tx=0x7f32dc005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a4f4b41eea354e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.425+0000 7f32d3e00700 1 -- 172.21.15.16:0/2351747992 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0053f0 con 0x55e4a3e94e70 2024-05-27T07:07:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.426+0000 7f32d3e00700 1 -- 172.21.15.16:0/2351747992 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc005590 con 0x55e4a3e94e70 2024-05-27T07:07:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.426+0000 7f32d3e00700 1 -- 172.21.15.16:0/2351747992 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004b10 con 0x55e4a3e94e70 2024-05-27T07:07:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.430+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2351747992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 msgr2=0x55e4a3e99f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.430+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32dc001eb0 tx=0x7f32dc005140 comp rx=0 tx=0).stop 2024-05-27T07:07:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.430+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2351747992 shutdown_connections 2024-05-27T07:07:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.430+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e99f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.431+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3d15e40 0x55e4a3e948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.431+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/2351747992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3e019f0 0x55e4a3e9d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.431+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2351747992 >> 172.21.15.16:0/2351747992 conn(0x55e4a3daf390 msgr2=0x55e4a3e941b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.432+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2351747992 shutdown_connections 2024-05-27T07:07:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.432+0000 7f32f93f8b80 1 -- 172.21.15.16:0/2351747992 wait complete. 2024-05-27T07:07:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.433+0000 7f32f93f8b80 1 -- start start 2024-05-27T07:07:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.433+0000 7f32f93f8b80 1 Processor -- start 2024-05-27T07:07:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.436+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 0x55e4a3e98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.436+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.436+0000 7f32f93f8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4074e30 con 0x55e4a3d15e40 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4074760 con 0x55e4a3e94e70 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32f93f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4071a10 con 0x55e4a3e019f0 2024-05-27T07:07:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32f93f8b80 10 client.0 _ll_register_callbacks cb 0x55e4a3d5cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32daa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32daa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e98a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33092/0 (socket says 172.21.15.16:33092) 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32daa00700 1 -- 172.21.15.16:0/3614323418 learned_addr learned my addr 172.21.15.16:0/3614323418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.437+0000 7f32e3e00700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3e00700 1 -- 172.21.15.16:0/3614323418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 msgr2=0x55e4a3e98a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3e00700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e98a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3400700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 0x55e4a3e98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3e00700 1 -- 172.21.15.16:0/3614323418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 msgr2=0x55e4a3e98520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3e00700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 0x55e4a3e98520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.438+0000 7f32e3e00700 1 -- 172.21.15.16:0/3614323418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a3eb69d0 con 0x55e4a3e94e70 2024-05-27T07:07:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.441+0000 7f32e3400700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 0x55e4a3e98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.441+0000 7f32e3e00700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e98fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f32dc01add0 tx=0x7f32dc0126c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d55181025e96fd86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.441+0000 7f32d1600700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:07:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.442+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc039060 con 0x55e4a3e94e70 2024-05-27T07:07:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.442+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f32dc01a620 con 0x55e4a3e94e70 2024-05-27T07:07:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.442+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc005590 con 0x55e4a3e94e70 2024-05-27T07:07:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.442+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32dc040050 con 0x55e4a3e94e70 2024-05-27T07:07:17.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.443+0000 7f32f93f8b80 10 client.36014 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:17.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.443+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4a3eb7bf0 con 0x55e4a3e94e70 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.443+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc005590 con 0x55e4a3e94e70 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.443+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f32dc010c00 con 0x55e4a3e94e70 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 10 client.36014 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 10 client.36014 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 20 client.36014 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3f2a600 0x55e4a3f2ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 4 client.36014 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:07:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.444+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4a3dbe570 con 0x55e4a3f2a600 2024-05-27T07:07:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.445+0000 7f32e3400700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3f2a600 0x55e4a3f2ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.447+0000 7f32d1600700 10 client.36014 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:07:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.447+0000 7f32e3400700 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3f2a600 0x55e4a3f2ca20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea2f84f302c88998 server_cookie=9c421f278110a3be in_seq=0 out_seq=0 2024-05-27T07:07:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.448+0000 7f32d1600700 1 -- 172.21.15.16:0/3614323418 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4a3dbe570 con 0x55e4a3f2a600 2024-05-27T07:07:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.448+0000 7f32d1600700 10 client.36014 handle_command_reply: tid=0 2024-05-27T07:07:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.448+0000 7f32f93f8b80 1 client.36014 shutdown 2024-05-27T07:07:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 msgr2=0x55e4a3e98fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e98fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f32dc01add0 tx=0x7f32dc0126c0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 shutdown_connections 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55e4a3f2a600 0x55e4a3f2ca20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a3e94e70 0x55e4a3e98fa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a3d15e40 0x55e4a3e98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 --2- 172.21.15.16:0/3614323418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4a3e019f0 0x55e4a3e98a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32d1600700 0 client.36014 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:07:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.449+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 >> 172.21.15.16:0/3614323418 conn(0x55e4a3daf390 msgr2=0x55e4a3e95ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.451+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 shutdown_connections 2024-05-27T07:07:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.451+0000 7f32f93f8b80 1 -- 172.21.15.16:0/3614323418 wait complete. 2024-05-27T07:07:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.451+0000 7f32f93f8b80 20 client.36014 trim_cache size 0 max 16384 2024-05-27T07:07:17.443 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 msgr2=0x7f32c80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 0x7f32c80b50d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32e41075c0 tx=0x7f32cc008040 comp rx=0 tx=0).stop 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 msgr2=0x7f32e419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f32d4000f80 tx=0x7f32d4004db0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 shutdown_connections 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f32c80b2bc0 0x7f32c80b50d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e4109840 0x7f32e419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32e41045c0 0x7f32e419adb0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f32d4000f80 tx=0x7f32d4004db0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 --2- 172.21.15.16:0/3265590418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e4108ef0 0x7f32e419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.453+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 >> 172.21.15.16:0/3265590418 conn(0x7f32e40ffe90 msgr2=0x7f32e4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.454+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 shutdown_connections 2024-05-27T07:07:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:17.454+0000 7f32ea800700 1 -- 172.21.15.16:0/3265590418 wait complete. 2024-05-27T07:07:17.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:17 smithi164 ceph-mon[35327]: pgmap v194: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:17.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:17 smithi016 ceph-mon[31415]: pgmap v194: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:17.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:17 smithi069 ceph-mon[35276]: pgmap v194: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:18.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:07:19.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:19 smithi016 ceph-mon[31415]: pgmap v195: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 514 op/s 2024-05-27T07:07:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:07:19.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:19 smithi069 ceph-mon[35276]: pgmap v195: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 514 op/s 2024-05-27T07:07:20.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:19 smithi164 ceph-mon[35327]: pgmap v195: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 514 op/s 2024-05-27T07:07:21.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:21 smithi016 ceph-mon[31415]: pgmap v196: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:07:21.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:21 smithi069 ceph-mon[35276]: pgmap v196: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:07:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:21 smithi164 ceph-mon[35327]: pgmap v196: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 MiB/s wr, 509 op/s 2024-05-27T07:07:23.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:23 smithi016 ceph-mon[31415]: pgmap v197: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:23.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:23 smithi069 ceph-mon[35276]: pgmap v197: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:23 smithi164 ceph-mon[35327]: pgmap v197: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 670 op/s 2024-05-27T07:07:25.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:25 smithi016 ceph-mon[31415]: pgmap v198: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:25.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:25 smithi069 ceph-mon[35276]: pgmap v198: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:26.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:25 smithi164 ceph-mon[35327]: pgmap v198: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:27.883 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:27 smithi164 ceph-mon[35327]: pgmap v199: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 696 op/s 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.051+0000 7f04c0c00700 1 -- 172.21.15.16:0/2609179863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc107de0 msgr2=0x7f04bc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.051+0000 7f04c0c00700 1 --2- 172.21.15.16:0/2609179863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc107de0 0x7f04bc1081f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009920 tx=0x7f04a4009c30 comp rx=0 tx=0).stop 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.052+0000 7f04c0c00700 1 -- 172.21.15.16:0/2609179863 shutdown_connections 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.052+0000 7f04c0c00700 1 --2- 172.21.15.16:0/2609179863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04bc108730 0x7f04bc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.052+0000 7f04c0c00700 1 --2- 172.21.15.16:0/2609179863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1034b0 0x7f04bc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.052+0000 7f04c0c00700 1 --2- 172.21.15.16:0/2609179863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc107de0 0x7f04bc1081f0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009920 tx=0x7f04a4009c30 comp rx=0 tx=0).stop 2024-05-27T07:07:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.052+0000 7f04c0c00700 1 -- 172.21.15.16:0/2609179863 >> 172.21.15.16:0/2609179863 conn(0x7f04bc0fedb0 msgr2=0x7f04bc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.054+0000 7f04c0c00700 1 -- 172.21.15.16:0/2609179863 shutdown_connections 2024-05-27T07:07:28.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:27 smithi016 ceph-mon[31415]: pgmap v199: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 696 op/s 2024-05-27T07:07:28.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.054+0000 7f04c0c00700 1 -- 172.21.15.16:0/2609179863 wait complete. 2024-05-27T07:07:28.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.055+0000 7f04c0c00700 1 Processor -- start 2024-05-27T07:07:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 -- start start 2024-05-27T07:07:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04bc108730 0x7f04bc19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 0x7f04bc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1132b0 con 0x7f04bc108730 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc113130 con 0x7f04bc1a1980 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.057+0000 7f04c0c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc113430 con 0x7f04bc1034b0 2024-05-27T07:07:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.058+0000 7f04b9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.058+0000 7f04b9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:57840/0 (socket says 172.21.15.16:57840) 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.058+0000 7f04b9e00700 1 -- 172.21.15.16:0/3249734881 learned_addr learned my addr 172.21.15.16:0/3249734881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.058+0000 7f04ba800700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 0x7f04bc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.059+0000 7f04b9e00700 1 -- 172.21.15.16:0/3249734881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 msgr2=0x7f04bc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.059+0000 7f04b9e00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 0x7f04bc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.059+0000 7f04b9e00700 1 -- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04bc108730 msgr2=0x7f04bc19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.059+0000 7f04b9e00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04bc108730 0x7f04bc19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.059+0000 7f04b9e00700 1 -- 172.21.15.16:0/3249734881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc1a24d0 con 0x7f04bc1034b0 2024-05-27T07:07:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.060+0000 7f04ba800700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 0x7f04bc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.060+0000 7f04b9e00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c0c0 tx=0x7f04ac00c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.061+0000 7f04b2a00700 1 -- 172.21.15.16:0/3249734881 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00cd10 con 0x7f04bc1034b0 2024-05-27T07:07:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.061+0000 7f04b2a00700 1 -- 172.21.15.16:0/3249734881 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04ac0075c0 con 0x7f04bc1034b0 2024-05-27T07:07:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.061+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1a2760 con 0x7f04bc1034b0 2024-05-27T07:07:28.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.061+0000 7f04b2a00700 1 -- 172.21.15.16:0/3249734881 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0146e0 con 0x7f04bc1034b0 2024-05-27T07:07:28.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.061+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1a9f90 con 0x7f04bc1034b0 2024-05-27T07:07:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.068+0000 7f04b2a00700 1 -- 172.21.15.16:0/3249734881 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f04ac01b020 con 0x7f04bc1034b0 2024-05-27T07:07:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.071+0000 7f04b2a00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 0x7f04a00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.071+0000 7f04b2a00700 1 -- 172.21.15.16:0/3249734881 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f04ac0e3670 con 0x7f04bc1034b0 2024-05-27T07:07:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.071+0000 7f04b3e00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 0x7f04a00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.094+0000 7f04b3e00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 0x7f04a00b4f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f04bc1077d0 tx=0x7f04a4005dc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.105+0000 7f04cf708b80 1 Processor -- start 2024-05-27T07:07:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.105+0000 7f04cf708b80 1 -- start start 2024-05-27T07:07:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.105+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65918e40 0x560f65aae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65aa8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 0x560f65aa91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04cf708b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f659c5e40 con 0x560f65aaed10 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04cf708b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f659b59e0 con 0x560f65aa8dc0 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04cf708b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f659b9a20 con 0x560f65918e40 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65aa8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65aa8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40800/0 (socket says 172.21.15.16:40800) 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.106+0000 7f04b3e00700 1 -- 172.21.15.16:0/2439376599 learned_addr learned my addr 172.21.15.16:0/2439376599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.107+0000 7f04ba800700 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 0x560f65aa91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04b9e00700 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65918e40 0x560f65aae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04ba800700 1 -- 172.21.15.16:0/2439376599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65918e40 msgr2=0x560f65aae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04ba800700 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65918e40 0x560f65aae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04ba800700 1 -- 172.21.15.16:0/2439376599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 msgr2=0x560f65aa8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04ba800700 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65aa8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.108+0000 7f04ba800700 1 -- 172.21.15.16:0/2439376599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6597e420 con 0x560f65aa8dc0 2024-05-27T07:07:28.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.109+0000 7f04ba800700 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 0x560f65aa91d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009dd0 tx=0x7f04b400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61effbce1d61993 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.110+0000 7f04ab400700 1 -- 172.21.15.16:0/2439376599 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4024070 con 0x560f65aa8dc0 2024-05-27T07:07:28.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.110+0000 7f04ab400700 1 -- 172.21.15.16:0/2439376599 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04b4017c60 con 0x560f65aa8dc0 2024-05-27T07:07:28.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.110+0000 7f04ab400700 1 -- 172.21.15.16:0/2439376599 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4024070 con 0x560f65aa8dc0 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 -- 172.21.15.16:0/2439376599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 msgr2=0x560f65aa91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 0x560f65aa91d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009dd0 tx=0x7f04b400e4b0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 -- 172.21.15.16:0/2439376599 shutdown_connections 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aa8dc0 0x560f65aa91d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65918e40 0x560f65aae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2439376599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65aa8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.111+0000 7f04cf708b80 1 -- 172.21.15.16:0/2439376599 >> 172.21.15.16:0/2439376599 conn(0x560f65978fc0 msgr2=0x560f65a928d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.112+0000 7f04cf708b80 1 -- 172.21.15.16:0/2439376599 shutdown_connections 2024-05-27T07:07:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.112+0000 7f04cf708b80 1 -- 172.21.15.16:0/2439376599 wait complete. 2024-05-27T07:07:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.113+0000 7f04cf708b80 1 -- start start 2024-05-27T07:07:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.113+0000 7f04cf708b80 1 Processor -- start 2024-05-27T07:07:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.114+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 0x560f65ab2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aaed10 0x560f65ab30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f658bbfa0 con 0x560f65aa8dc0 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f65956760 con 0x560f65918e40 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f65ab5810 con 0x560f65aaed10 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04cf708b80 10 client.0 _ll_register_callbacks cb 0x560f6595fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04b9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04b9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42988/0 (socket says 172.21.15.16:42988) 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04b9e00700 1 -- 172.21.15.16:0/1665595628 learned_addr learned my addr 172.21.15.16:0/1665595628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.115+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 0x560f65ab2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b9e00700 1 -- 172.21.15.16:0/1665595628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aaed10 msgr2=0x560f65ab30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b9e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aaed10 0x560f65ab30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b9e00700 1 -- 172.21.15.16:0/1665595628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 msgr2=0x560f65ab2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b9e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 0x560f65ab2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b9e00700 1 -- 172.21.15.16:0/1665595628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f659058d0 con 0x560f65918e40 2024-05-27T07:07:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.116+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 0x560f65ab2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.117+0000 7f04b9e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0e1030 tx=0x7f04ac0aa020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a175d61af8a225f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.117+0000 7f049fe00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.117+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0c4060 con 0x560f65918e40 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.117+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f04ac0aa820 con 0x560f65918e40 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.117+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b4017c60 con 0x560f65918e40 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.118+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f04ac0c9050 con 0x560f65918e40 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.118+0000 7f04cf708b80 10 client.36026 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.118+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560f65986060 con 0x560f65918e40 2024-05-27T07:07:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.118+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04b4017c60 con 0x560f65918e40 2024-05-27T07:07:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f04ac0a4ce0 con 0x560f65918e40 2024-05-27T07:07:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 10 client.36026 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 10 client.36026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:07:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 20 client.36026 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65ac2580 0x560f65ac49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 4 client.36026 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:07:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.119+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f659c7d90 con 0x560f65ac2580 2024-05-27T07:07:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.120+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65ac2580 0x560f65ac49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.122+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65ac2580 0x560f65ac49a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e77ff1b3e3aeb9a1 server_cookie=464f2e9ed7bfa27 in_seq=0 out_seq=0 2024-05-27T07:07:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.122+0000 7f049fe00700 10 client.36026 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:07:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.124+0000 7f049fe00700 1 -- 172.21.15.16:0/1665595628 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560f659c7d90 con 0x560f65ac2580 2024-05-27T07:07:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.124+0000 7f049fe00700 10 client.36026 handle_command_reply: tid=0 2024-05-27T07:07:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.125+0000 7f04cf708b80 1 client.36026 shutdown 2024-05-27T07:07:28.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:27 smithi069 ceph-mon[35276]: pgmap v199: 129 pgs: 129 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 696 op/s 2024-05-27T07:07:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.127+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 msgr2=0x560f65ab2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.127+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0e1030 tx=0x7f04ac0aa020 comp rx=0 tx=0).stop 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 shutdown_connections 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65ac2580 0x560f65ac49a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aaed10 0x560f65ab30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65ab2630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1665595628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aa8dc0 0x560f65ab2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 >> 172.21.15.16:0/1665595628 conn(0x560f65978fc0 msgr2=0x560f65a92580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.128+0000 7f049fe00700 0 client.36026 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:07:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.129+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 shutdown_connections 2024-05-27T07:07:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.130+0000 7f04cf708b80 1 -- 172.21.15.16:0/1665595628 wait complete. 2024-05-27T07:07:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.130+0000 7f04cf708b80 20 client.36026 trim_cache size 0 max 16384 2024-05-27T07:07:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.166+0000 7f04cf708b80 1 Processor -- start 2024-05-27T07:07:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.166+0000 7f04cf708b80 1 -- start start 2024-05-27T07:07:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.166+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 0x560f65ab1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.167+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.167+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.167+0000 7f04cf708b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc071c30 con 0x560f65918e40 2024-05-27T07:07:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.167+0000 7f04cf708b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc073770 con 0x560f65aaed10 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.167+0000 7f04cf708b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc072300 con 0x560f65aa8dc0 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04ba800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04ba800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42990/0 (socket says 172.21.15.16:42990) 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:57858/0 (socket says 172.21.15.16:57858) 2024-05-27T07:07:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04b3e00700 1 -- 172.21.15.16:0/1305555756 learned_addr learned my addr 172.21.15.16:0/1305555756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:28.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.168+0000 7f04b9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 0x560f65ab1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.169+0000 7f04b3e00700 1 -- 172.21.15.16:0/1305555756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 msgr2=0x560f65aac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.170+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.170+0000 7f04b3e00700 1 -- 172.21.15.16:0/1305555756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 msgr2=0x560f65ab1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.170+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 0x560f65ab1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.170+0000 7f04b3e00700 1 -- 172.21.15.16:0/1305555756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f65aae190 con 0x560f65aa8dc0 2024-05-27T07:07:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.170+0000 7f04b9e00700 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 0x560f65ab1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.171+0000 7f04ba800700 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.173+0000 7f04b3e00700 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009920 tx=0x7f04a4023040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9890ee90442f9fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.173+0000 7f04aaa00700 1 -- 172.21.15.16:0/1305555756 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4005040 con 0x560f65aa8dc0 2024-05-27T07:07:28.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.174+0000 7f04aaa00700 1 -- 172.21.15.16:0/1305555756 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a4005670 con 0x560f65aa8dc0 2024-05-27T07:07:28.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.174+0000 7f04aaa00700 1 -- 172.21.15.16:0/1305555756 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4003d50 con 0x560f65aa8dc0 2024-05-27T07:07:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.176+0000 7f04cf708b80 1 -- 172.21.15.16:0/1305555756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 msgr2=0x560f65aac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.176+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009920 tx=0x7f04a4023040 comp rx=0 tx=0).stop 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 -- 172.21.15.16:0/1305555756 shutdown_connections 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65aaed10 0x560f65aac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65918e40 0x560f65ab1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 --2- 172.21.15.16:0/1305555756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65aac190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 -- 172.21.15.16:0/1305555756 >> 172.21.15.16:0/1305555756 conn(0x560f65978fc0 msgr2=0x560f65ab0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 -- 172.21.15.16:0/1305555756 shutdown_connections 2024-05-27T07:07:28.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.178+0000 7f04cf708b80 1 -- 172.21.15.16:0/1305555756 wait complete. 2024-05-27T07:07:28.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.179+0000 7f04cf708b80 1 -- start start 2024-05-27T07:07:28.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.179+0000 7f04cf708b80 1 Processor -- start 2024-05-27T07:07:28.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65abad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65abb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65ac9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0730a0 con 0x560f65aaed10 2024-05-27T07:07:28.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0729d0 con 0x560f65918e40 2024-05-27T07:07:28.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc06fb90 con 0x560f65aa8dc0 2024-05-27T07:07:28.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.181+0000 7f04cf708b80 10 client.0 _ll_register_callbacks cb 0x560f6595fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:28.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.182+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65abb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.182+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65abb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:57860/0 (socket says 172.21.15.16:57860) 2024-05-27T07:07:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.182+0000 7f04b3e00700 1 -- 172.21.15.16:0/2458330932 learned_addr learned my addr 172.21.15.16:0/2458330932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.182+0000 7f04b9e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65abad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.183+0000 7f04ba800700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65ac9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.183+0000 7f04b9e00700 1 -- 172.21.15.16:0/2458330932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 msgr2=0x560f65abb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.183+0000 7f04b9e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65abb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.183+0000 7f04b9e00700 1 -- 172.21.15.16:0/2458330932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 msgr2=0x560f65ac9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.183+0000 7f04b9e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65ac9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.184+0000 7f04b9e00700 1 -- 172.21.15.16:0/2458330932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f65ac9f80 con 0x560f65918e40 2024-05-27T07:07:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.185+0000 7f04b9e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65abad80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0aa470 tx=0x7f04ac0aabe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc810a8705c62a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.185+0000 7f049fe00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:07:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.185+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac088070 con 0x560f65918e40 2024-05-27T07:07:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.185+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f04ac0a9820 con 0x560f65918e40 2024-05-27T07:07:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.185+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a4005670 con 0x560f65918e40 2024-05-27T07:07:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.186+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f04ac0c9050 con 0x560f65918e40 2024-05-27T07:07:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.186+0000 7f04cf708b80 10 client.36032 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.186+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560f65b2caf0 con 0x560f65918e40 2024-05-27T07:07:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.186+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a4005670 con 0x560f65918e40 2024-05-27T07:07:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.187+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f04ac0a4ce0 con 0x560f65918e40 2024-05-27T07:07:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.187+0000 7f04cf708b80 10 client.36032 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.187+0000 7f04cf708b80 10 client.36032 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.188+0000 7f04cf708b80 20 client.36032 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.188+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65b2d230 0x560f65b2f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.188+0000 7f04cf708b80 4 client.36032 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.188+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f659c1570 con 0x560f65b2d230 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.188+0000 7f04b3e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65b2d230 0x560f65b2f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.190+0000 7f04b3e00700 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65b2d230 0x560f65b2f650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10416062a738964c server_cookie=f55f24c49d54fc20 in_seq=0 out_seq=0 2024-05-27T07:07:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.190+0000 7f049fe00700 10 client.36032 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:07:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.191+0000 7f049fe00700 1 -- 172.21.15.16:0/2458330932 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f659c1570 con 0x560f65b2d230 2024-05-27T07:07:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.191+0000 7f049fe00700 10 client.36032 handle_command_reply: tid=0 2024-05-27T07:07:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.191+0000 7f04cf708b80 1 client.36032 shutdown 2024-05-27T07:07:28.192 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:07:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 msgr2=0x560f65abad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65abad80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0aa470 tx=0x7f04ac0aabe0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 shutdown_connections 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560f65b2d230 0x560f65b2f650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f65aaed10 0x560f65ac9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f65918e40 0x560f65abad80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 --2- 172.21.15.16:0/2458330932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f65aa8dc0 0x560f65abb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 >> 172.21.15.16:0/2458330932 conn(0x560f65978fc0 msgr2=0x560f65a96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.192+0000 7f049fe00700 0 client.36032 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.193+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 shutdown_connections 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.193+0000 7f04cf708b80 1 -- 172.21.15.16:0/2458330932 wait complete. 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.193+0000 7f04cf708b80 20 client.36032 trim_cache size 0 max 16384 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.195+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 msgr2=0x7f04a00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.195+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 0x7f04a00b4f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f04bc1077d0 tx=0x7f04a4005dc0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.195+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 msgr2=0x7f04bc19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.195+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c0c0 tx=0x7f04ac00c3d0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 shutdown_connections 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f04a00b2a70 0x7f04a00b4f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04bc1a1980 0x7f04bc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04bc1034b0 0x7f04bc19d210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 --2- 172.21.15.16:0/3249734881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04bc108730 0x7f04bc19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.196+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 >> 172.21.15.16:0/3249734881 conn(0x7f04bc0fedb0 msgr2=0x7f04bc1066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.197+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 shutdown_connections 2024-05-27T07:07:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:28.197+0000 7f04c0c00700 1 -- 172.21.15.16:0/3249734881 wait complete. 2024-05-27T07:07:29.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:07:29.920 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:29 smithi069 ceph-mon[35276]: pgmap v200: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:29.920 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:29 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:07:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:29 smithi164 ceph-mon[35327]: pgmap v200: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:29 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:07:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:29 smithi016 ceph-mon[31415]: pgmap v200: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 32 MiB/s wr, 517 op/s 2024-05-27T07:07:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:29 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:07:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:07:32.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:31 smithi016 ceph-mon[31415]: pgmap v201: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:31 smithi164 ceph-mon[35327]: pgmap v201: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:32.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:31 smithi069 ceph-mon[35276]: pgmap v201: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:33.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:33 smithi016 ceph-mon[31415]: pgmap v202: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 658 op/s 2024-05-27T07:07:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:33 smithi164 ceph-mon[35327]: pgmap v202: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 658 op/s 2024-05-27T07:07:34.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:33 smithi069 ceph-mon[35276]: pgmap v202: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 658 op/s 2024-05-27T07:07:35.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:34 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:07:35.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:34 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:07:35.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:34 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:07:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:34 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:07:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:34 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:07:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:34 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:07:35.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:34 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:07:35.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:34 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:07:35.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:34 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:07:35.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:35 smithi016 ceph-mon[31415]: pgmap v203: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 31 MiB/s wr, 511 op/s 2024-05-27T07:07:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:35 smithi164 ceph-mon[35327]: pgmap v203: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 31 MiB/s wr, 511 op/s 2024-05-27T07:07:36.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:35 smithi069 ceph-mon[35276]: pgmap v203: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 31 MiB/s wr, 511 op/s 2024-05-27T07:07:38.047 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:37 smithi016 ceph-mon[31415]: pgmap v204: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 672 op/s 2024-05-27T07:07:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:37 smithi164 ceph-mon[35327]: pgmap v204: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 672 op/s 2024-05-27T07:07:38.121 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:37 smithi069 ceph-mon[35276]: pgmap v204: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 672 op/s 2024-05-27T07:07:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:39 smithi164 ceph-mon[35327]: pgmap v205: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:07:40.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:39 smithi016 ceph-mon[31415]: pgmap v205: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:07:40.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:39 smithi069 ceph-mon[35276]: pgmap v205: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 495 op/s 2024-05-27T07:07:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.821+0000 7fd3f9400700 1 -- 172.21.15.16:0/2323595704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 msgr2=0x7fd3f4083890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.821+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2323595704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 0x7fd3f4083890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0099c0 tx=0x7fd3ec009cd0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 -- 172.21.15.16:0/2323595704 shutdown_connections 2024-05-27T07:07:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2323595704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 0x7fd3f4083890 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2323595704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f407e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2323595704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f407ed60 0x7fd3f4082f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 -- 172.21.15.16:0/2323595704 >> 172.21.15.16:0/2323595704 conn(0x7fd3f407a010 msgr2=0x7fd3f407c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 -- 172.21.15.16:0/2323595704 shutdown_connections 2024-05-27T07:07:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.822+0000 7fd3f9400700 1 -- 172.21.15.16:0/2323595704 wait complete. 2024-05-27T07:07:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.826+0000 7fd3f9400700 1 Processor -- start 2024-05-27T07:07:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 -- start start 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f407ed60 0x7fd3f406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 0x7fd3f406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4123240 con 0x7fd3f4083480 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41230c0 con 0x7fd3f407ed60 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.827+0000 7fd3f9400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41233c0 con 0x7fd3f407e410 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.828+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.828+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33240/0 (socket says 172.21.15.16:33240) 2024-05-27T07:07:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.828+0000 7fd3f2800700 1 -- 172.21.15.16:0/2754529619 learned_addr learned my addr 172.21.15.16:0/2754529619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 -- 172.21.15.16:0/2754529619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f407ed60 msgr2=0x7fd3f406e2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f407ed60 0x7fd3f406e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 -- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 msgr2=0x7fd3f406e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 0x7fd3f406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 -- 172.21.15.16:0/2754529619 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f406ef10 con 0x7fd3f407e410 2024-05-27T07:07:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.829+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400efc0 tx=0x7fd3e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.830+0000 7fd3ebe00700 1 -- 172.21.15.16:0/2754529619 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400cad0 con 0x7fd3f407e410 2024-05-27T07:07:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.830+0000 7fd3ebe00700 1 -- 172.21.15.16:0/2754529619 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3e4019070 con 0x7fd3f407e410 2024-05-27T07:07:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.830+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f4072ca0 con 0x7fd3f407e410 2024-05-27T07:07:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.830+0000 7fd3ebe00700 1 -- 172.21.15.16:0/2754529619 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4014480 con 0x7fd3f407e410 2024-05-27T07:07:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.830+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f40731e0 con 0x7fd3f407e410 2024-05-27T07:07:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.836+0000 7fd3ebe00700 1 -- 172.21.15.16:0/2754529619 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd3e4012050 con 0x7fd3f407e410 2024-05-27T07:07:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.839+0000 7fd3ebe00700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 0x7fd3d80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.839+0000 7fd3ebe00700 1 -- 172.21.15.16:0/2754529619 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3e40e2930 con 0x7fd3f407e410 2024-05-27T07:07:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.840+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 0x7fd3d80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.851+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 0x7fd3d80b50d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3f406aaf0 tx=0x7fd3dc006c60 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.855+0000 7fd407f4ab80 1 Processor -- start 2024-05-27T07:07:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.856+0000 7fd407f4ab80 1 -- start start 2024-05-27T07:07:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.858+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.858+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74acbd30 0x555b74ac5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74ac61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b749f3e40 con 0x555b74acbd30 2024-05-27T07:07:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555b749e39e0 con 0x555b74ac5db0 2024-05-27T07:07:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555b749e7a20 con 0x555b74946e40 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33254/0 (socket says 172.21.15.16:33254) 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd3f2800700 1 -- 172.21.15.16:0/333831591 learned_addr learned my addr 172.21.15.16:0/333831591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd3f3200700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74ac61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.859+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74acbd30 0x555b74ac5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f3200700 1 -- 172.21.15.16:0/333831591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 msgr2=0x555b74acb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f3200700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f3200700 1 -- 172.21.15.16:0/333831591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74acbd30 msgr2=0x555b74ac5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f3200700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74acbd30 0x555b74ac5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f3200700 1 -- 172.21.15.16:0/333831591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b749ac420 con 0x555b74ac5db0 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.861+0000 7fd3f2800700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.862+0000 7fd3f3200700 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74ac61c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009990 tx=0x7fd3ec009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7b5bafb9980d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.862+0000 7fd3e3e00700 1 -- 172.21.15.16:0/333831591 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec005450 con 0x555b74ac5db0 2024-05-27T07:07:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.863+0000 7fd3e3e00700 1 -- 172.21.15.16:0/333831591 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3ec0055f0 con 0x555b74ac5db0 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 -- 172.21.15.16:0/333831591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 msgr2=0x555b74ac61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74ac61c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009990 tx=0x7fd3ec009300 comp rx=0 tx=0).stop 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 -- 172.21.15.16:0/333831591 shutdown_connections 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74ac61c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74acb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.865+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/333831591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74acbd30 0x555b74ac5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.866+0000 7fd407f4ab80 1 -- 172.21.15.16:0/333831591 >> 172.21.15.16:0/333831591 conn(0x555b749a6fc0 msgr2=0x555b74ac08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.868+0000 7fd407f4ab80 1 -- 172.21.15.16:0/333831591 shutdown_connections 2024-05-27T07:07:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.868+0000 7fd407f4ab80 1 -- 172.21.15.16:0/333831591 wait complete. 2024-05-27T07:07:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.869+0000 7fd407f4ab80 1 -- start start 2024-05-27T07:07:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.869+0000 7fd407f4ab80 1 Processor -- start 2024-05-27T07:07:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.871+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 0x555b74acf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74acfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b748e9fa0 con 0x555b74946e40 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555b74984760 con 0x555b74ac5db0 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd3e9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.875+0000 7fd3e9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:55072/0 (socket says 172.21.15.16:55072) 2024-05-27T07:07:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd3f3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74acfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd3f3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74acfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33270/0 (socket says 172.21.15.16:33270) 2024-05-27T07:07:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd3e9e00700 1 -- 172.21.15.16:0/2835499478 learned_addr learned my addr 172.21.15.16:0/2835499478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555b74ad27f0 con 0x555b74acbd30 2024-05-27T07:07:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd407f4ab80 10 client.0 _ll_register_callbacks cb 0x555b7498dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.876+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 0x555b74acf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.877+0000 7fd3e9e00700 1 -- 172.21.15.16:0/2835499478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 msgr2=0x555b74acfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.877+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74acfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3e9e00700 1 -- 172.21.15.16:0/2835499478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 msgr2=0x555b74acf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 0x555b74acf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3e9e00700 1 -- 172.21.15.16:0/2835499478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b749338d0 con 0x555b74ac5db0 2024-05-27T07:07:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 0x555b74acf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:07:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012660 tx=0x7fd3dc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a55c07f3e9a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.879+0000 7fd3e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:07:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.881+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc024060 con 0x555b74ac5db0 2024-05-27T07:07:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.881+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3dc010770 con 0x555b74ac5db0 2024-05-27T07:07:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.881+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ec0055f0 con 0x555b74ac5db0 2024-05-27T07:07:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.882+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc02b050 con 0x555b74ac5db0 2024-05-27T07:07:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.882+0000 7fd407f4ab80 10 client.36038 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.882+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b749b4060 con 0x555b74ac5db0 2024-05-27T07:07:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.882+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3ec0055f0 con 0x555b74ac5db0 2024-05-27T07:07:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd3dc01e070 con 0x555b74ac5db0 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 10 client.36038 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 10 client.36038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 20 client.36038 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74adf6e0 0x555b74ae1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 4 client.36038 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b749f5d90 con 0x555b74adf6e0 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.883+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74adf6e0 0x555b74ae1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.884+0000 7fd3f2800700 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74adf6e0 0x555b74ae1b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6da43ea9bd8c9234 server_cookie=bbf020eb3444d48b in_seq=0 out_seq=0 2024-05-27T07:07:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.884+0000 7fd3e0c00700 10 client.36038 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:07:40.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.887+0000 7fd3e0c00700 1 -- 172.21.15.16:0/2835499478 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555b749f5d90 con 0x555b74adf6e0 2024-05-27T07:07:40.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.887+0000 7fd3e0c00700 10 client.36038 handle_command_reply: tid=0 2024-05-27T07:07:40.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.887+0000 7fd407f4ab80 1 client.36038 shutdown 2024-05-27T07:07:40.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 msgr2=0x555b74acf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012660 tx=0x7fd3dc004770 comp rx=0 tx=0).stop 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 shutdown_connections 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74adf6e0 0x555b74ae1b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74acfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74946e40 0x555b74acf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/2835499478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74ac5db0 0x555b74acf980 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 >> 172.21.15.16:0/2835499478 conn(0x555b749a6fc0 msgr2=0x555b74ac0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd3e0c00700 0 client.36038 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 shutdown_connections 2024-05-27T07:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.888+0000 7fd407f4ab80 1 -- 172.21.15.16:0/2835499478 wait complete. 2024-05-27T07:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.889+0000 7fd407f4ab80 20 client.36038 trim_cache size 0 max 16384 2024-05-27T07:07:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.923+0000 7fd407f4ab80 1 Processor -- start 2024-05-27T07:07:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.923+0000 7fd407f4ab80 1 -- start start 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 0x555b74adb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74acfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c7a60 con 0x555b74ad1a50 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c95a0 con 0x555b74acbd30 2024-05-27T07:07:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.924+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c8130 con 0x555b74946e40 2024-05-27T07:07:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.926+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.926+0000 7fd3e9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 0x555b74adb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.926+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33284/0 (socket says 172.21.15.16:33284) 2024-05-27T07:07:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 -- 172.21.15.16:0/3687805562 learned_addr learned my addr 172.21.15.16:0/3687805562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.926+0000 7fd3f3200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74acfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.926+0000 7fd3e9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 0x555b74adb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:55078/0 (socket says 172.21.15.16:55078) 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 -- 172.21.15.16:0/3687805562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 msgr2=0x555b74adb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 0x555b74adb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 -- 172.21.15.16:0/3687805562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 msgr2=0x555b74acfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74acfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.927+0000 7fd3f2800700 1 -- 172.21.15.16:0/3687805562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b74add100 con 0x555b74946e40 2024-05-27T07:07:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.928+0000 7fd3f2800700 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3e401d000 tx=0x7fd3e40cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b174c5260ca63233 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.929+0000 7fd3e3400700 1 -- 172.21.15.16:0/3687805562 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40c3070 con 0x555b74946e40 2024-05-27T07:07:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.929+0000 7fd3e3400700 1 -- 172.21.15.16:0/3687805562 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3e40adbd0 con 0x555b74946e40 2024-05-27T07:07:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.929+0000 7fd3e3400700 1 -- 172.21.15.16:0/3687805562 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40a9c00 con 0x555b74946e40 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.932+0000 7fd407f4ab80 1 -- 172.21.15.16:0/3687805562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 msgr2=0x555b74ac5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.932+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3e401d000 tx=0x7fd3e40cc040 comp rx=0 tx=0).stop 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 -- 172.21.15.16:0/3687805562 shutdown_connections 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74acfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74946e40 0x555b74ac5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/3687805562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74acbd30 0x555b74adb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 -- 172.21.15.16:0/3687805562 >> 172.21.15.16:0/3687805562 conn(0x555b749a6fc0 msgr2=0x555b74acf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 -- 172.21.15.16:0/3687805562 shutdown_connections 2024-05-27T07:07:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.933+0000 7fd407f4ab80 1 -- 172.21.15.16:0/3687805562 wait complete. 2024-05-27T07:07:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.934+0000 7fd407f4ab80 1 -- start start 2024-05-27T07:07:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.935+0000 7fd407f4ab80 1 Processor -- start 2024-05-27T07:07:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 0x555b74ae7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74ae75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74ae7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c8ed0 con 0x555b74ad1a50 2024-05-27T07:07:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c8800 con 0x555b74946e40 2024-05-27T07:07:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41c5a00 con 0x555b74acbd30 2024-05-27T07:07:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.936+0000 7fd407f4ab80 10 client.0 _ll_register_callbacks cb 0x555b74ac68f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.937+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 0x555b74ae7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.937+0000 7fd3f2800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 0x555b74ae7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:55092/0 (socket says 172.21.15.16:55092) 2024-05-27T07:07:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.937+0000 7fd3f2800700 1 -- 172.21.15.16:0/1815105354 learned_addr learned my addr 172.21.15.16:0/1815105354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.937+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74ae75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.938+0000 7fd3e9e00700 1 -- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 msgr2=0x555b74ae7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.938+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 0x555b74ae7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.938+0000 7fd3e9e00700 1 -- 172.21.15.16:0/1815105354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 msgr2=0x555b74ae7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.938+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74ae7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.938+0000 7fd3e9e00700 1 -- 172.21.15.16:0/1815105354 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b74b49380 con 0x555b74acbd30 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.939+0000 7fd3e9e00700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74ae75a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x555b74ad0500 tx=0x7fd3dc01f620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1612d44f04162421 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.939+0000 7fd3e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:07:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.941+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc033070 con 0x555b74acbd30 2024-05-27T07:07:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.941+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd3dc004020 con 0x555b74acbd30 2024-05-27T07:07:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.941+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e40adbd0 con 0x555b74acbd30 2024-05-27T07:07:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.942+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc01fb20 con 0x555b74acbd30 2024-05-27T07:07:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.942+0000 7fd407f4ab80 10 client.25981 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.942+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b74b4a5c0 con 0x555b74acbd30 2024-05-27T07:07:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.943+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3e40adbd0 con 0x555b74acbd30 2024-05-27T07:07:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.947+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd3dc036030 con 0x555b74acbd30 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 10 client.25981 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 10 client.25981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 20 client.25981 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74b4ac10 0x555b74b4d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 4 client.25981 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b749ef570 con 0x555b74b4ac10 2024-05-27T07:07:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.948+0000 7fd3f2800700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74b4ac10 0x555b74b4d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.952+0000 7fd3f2800700 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74b4ac10 0x555b74b4d030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0f6d8b5784247ad server_cookie=cde31cca3b651411 in_seq=0 out_seq=0 2024-05-27T07:07:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd3e0c00700 10 client.25981 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:07:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd3e0c00700 1 -- 172.21.15.16:0/1815105354 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b749ef570 con 0x555b74b4ac10 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd3e0c00700 10 client.25981 handle_command_reply: tid=0 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 client.25981 shutdown 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 msgr2=0x555b74ae75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74ae75a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x555b74ad0500 tx=0x7fd3dc01f620 comp rx=0 tx=0).stop 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 shutdown_connections 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x555b74b4ac10 0x555b74b4d030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b74ad1a50 0x555b74ae7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b74946e40 0x555b74ae7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 --2- 172.21.15.16:0/1815105354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b74acbd30 0x555b74ae75a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd3e0c00700 0 client.25981 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:07:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.953+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 >> 172.21.15.16:0/1815105354 conn(0x555b749a6fc0 msgr2=0x555b74ac99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.954+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 shutdown_connections 2024-05-27T07:07:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.954+0000 7fd407f4ab80 1 -- 172.21.15.16:0/1815105354 wait complete. 2024-05-27T07:07:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.955+0000 7fd407f4ab80 20 client.25981 trim_cache size 0 max 16384 2024-05-27T07:07:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:07:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 msgr2=0x7fd3d80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 0x7fd3d80b50d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3f406aaf0 tx=0x7fd3dc006c60 comp rx=0 tx=0).stop 2024-05-27T07:07:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 msgr2=0x7fd3f4075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400efc0 tx=0x7fd3e400c3a0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 shutdown_connections 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3d80b2bc0 0x7fd3d80b50d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4083480 0x7fd3f406e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f407e410 0x7fd3f4075170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 --2- 172.21.15.16:0/2754529619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f407ed60 0x7fd3f406e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.958+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 >> 172.21.15.16:0/2754529619 conn(0x7fd3f407a010 msgr2=0x7fd3f4085fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.959+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 shutdown_connections 2024-05-27T07:07:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:40.959+0000 7fd3f9400700 1 -- 172.21.15.16:0/2754529619 wait complete. 2024-05-27T07:07:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:41 smithi016 ceph-mon[31415]: pgmap v206: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 484 op/s 2024-05-27T07:07:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:41 smithi164 ceph-mon[35327]: pgmap v206: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 484 op/s 2024-05-27T07:07:42.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:41 smithi069 ceph-mon[35276]: pgmap v206: 129 pgs: 129 active+clean; 15 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 484 op/s 2024-05-27T07:07:43.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:07:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:43 smithi164 ceph-mon[35327]: pgmap v207: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 639 op/s 2024-05-27T07:07:44.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:43 smithi016 ceph-mon[31415]: pgmap v207: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 639 op/s 2024-05-27T07:07:44.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:43 smithi069 ceph-mon[35276]: pgmap v207: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 639 op/s 2024-05-27T07:07:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:07:45.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:45 smithi016 ceph-mon[31415]: pgmap v208: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 30 MiB/s wr, 487 op/s 2024-05-27T07:07:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:45 smithi164 ceph-mon[35327]: pgmap v208: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 30 MiB/s wr, 487 op/s 2024-05-27T07:07:46.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:45 smithi069 ceph-mon[35276]: pgmap v208: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 30 MiB/s wr, 487 op/s 2024-05-27T07:07:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:47 smithi164 ceph-mon[35327]: pgmap v209: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 40 MiB/s wr, 655 op/s 2024-05-27T07:07:48.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:47 smithi016 ceph-mon[31415]: pgmap v209: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 40 MiB/s wr, 655 op/s 2024-05-27T07:07:48.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:47 smithi069 ceph-mon[35276]: pgmap v209: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 40 MiB/s wr, 655 op/s 2024-05-27T07:07:50.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:49 smithi164 ceph-mon[35327]: pgmap v210: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 461 op/s 2024-05-27T07:07:50.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:49 smithi016 ceph-mon[31415]: pgmap v210: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 461 op/s 2024-05-27T07:07:50.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:49 smithi069 ceph-mon[35276]: pgmap v210: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 461 op/s 2024-05-27T07:07:51.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:51 smithi016 ceph-mon[31415]: pgmap v211: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 457 op/s 2024-05-27T07:07:52.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:51 smithi164 ceph-mon[35327]: pgmap v211: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 457 op/s 2024-05-27T07:07:52.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:51 smithi069 ceph-mon[35276]: pgmap v211: 129 pgs: 129 active+clean; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 28 MiB/s wr, 457 op/s 2024-05-27T07:07:53.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 -- 172.21.15.16:0/775514516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010ac00 msgr2=0x7f1b0010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 --2- 172.21.15.16:0/775514516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010ac00 0x7f1b0010b010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0074e0 tx=0x7f1aec00af90 comp rx=0 tx=0).stop 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 -- 172.21.15.16:0/775514516 shutdown_connections 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 --2- 172.21.15.16:0/775514516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010b550 0x7f1b001179b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 --2- 172.21.15.16:0/775514516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00110ba0 0x7f1b00110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 --2- 172.21.15.16:0/775514516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010ac00 0x7f1b0010b010 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 -- 172.21.15.16:0/775514516 >> 172.21.15.16:0/775514516 conn(0x7f1b0006f280 msgr2=0x7f1b0006f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.114+0000 7f1b06000700 1 -- 172.21.15.16:0/775514516 shutdown_connections 2024-05-27T07:07:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.115+0000 7f1b06000700 1 -- 172.21.15.16:0/775514516 wait complete. 2024-05-27T07:07:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.116+0000 7f1b06000700 1 Processor -- start 2024-05-27T07:07:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.116+0000 7f1b06000700 1 -- start start 2024-05-27T07:07:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010ac00 0x7f1b0011d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00110ba0 0x7f1b00124250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00119f50 con 0x7f1b00110ba0 2024-05-27T07:07:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00119dd0 con 0x7f1b0010ac00 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.118+0000 7f1b06000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0011a0d0 con 0x7f1b0010b550 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60096/0 (socket says 172.21.15.16:60096) 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 -- 172.21.15.16:0/4154700395 learned_addr learned my addr 172.21.15.16:0/4154700395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 -- 172.21.15.16:0/4154700395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010ac00 msgr2=0x7f1b0011d1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010ac00 0x7f1b0011d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 -- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00110ba0 msgr2=0x7f1b00124250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00110ba0 0x7f1b00124250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.119+0000 7f1affe00700 1 -- 172.21.15.16:0/4154700395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b00124790 con 0x7f1b0010b550 2024-05-27T07:07:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.120+0000 7f1affe00700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005b40 tx=0x7f1aec003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.121+0000 7f1afd600700 1 -- 172.21.15.16:0/4154700395 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec018070 con 0x7f1b0010b550 2024-05-27T07:07:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.121+0000 7f1afd600700 1 -- 172.21.15.16:0/4154700395 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1aec007d20 con 0x7f1b0010b550 2024-05-27T07:07:53.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.121+0000 7f1afd600700 1 -- 172.21.15.16:0/4154700395 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00a3c0 con 0x7f1b0010b550 2024-05-27T07:07:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.122+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b00124a20 con 0x7f1b0010b550 2024-05-27T07:07:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.122+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b00124f10 con 0x7f1b0010b550 2024-05-27T07:07:53.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.133+0000 7f1afd600700 1 -- 172.21.15.16:0/4154700395 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1aec00cae0 con 0x7f1b0010b550 2024-05-27T07:07:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.135+0000 7f1afd600700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 0x7f1ae40b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.136+0000 7f1afd600700 1 -- 172.21.15.16:0/4154700395 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1aec010020 con 0x7f1b0010b550 2024-05-27T07:07:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.136+0000 7f1b04c00700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 0x7f1ae40b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.151+0000 7f1b04c00700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 0x7f1ae40b4f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1af000ab90 tx=0x7f1af0009250 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.153+0000 7f1b160d1b80 1 Processor -- start 2024-05-27T07:07:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.154+0000 7f1b160d1b80 1 -- start start 2024-05-27T07:07:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.155+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 0x563ef0305ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.156+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.156+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 0x563ef0313020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.156+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ef0234e40 con 0x563ef0187e40 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.156+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ef02249e0 con 0x563ef0306820 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.157+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 0x563ef0305ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.157+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 0x563ef0305ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59512/0 (socket says 172.21.15.16:59512) 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.157+0000 7f1b04c00700 1 -- 172.21.15.16:0/2746502280 learned_addr learned my addr 172.21.15.16:0/2746502280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.157+0000 7f1b05600700 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 0x563ef0313020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.157+0000 7f1affe00700 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.158+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ef0228a20 con 0x563ef0306410 2024-05-27T07:07:53.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.159+0000 7f1b05600700 1 -- 172.21.15.16:0/2746502280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 msgr2=0x563ef0310ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.159+0000 7f1b05600700 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.159+0000 7f1b05600700 1 -- 172.21.15.16:0/2746502280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 msgr2=0x563ef0305ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.159+0000 7f1b05600700 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 0x563ef0305ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.159+0000 7f1b05600700 1 -- 172.21.15.16:0/2746502280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef01ed420 con 0x563ef0306820 2024-05-27T07:07:53.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.160+0000 7f1b05600700 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 0x563ef0313020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009bd0 tx=0x7f1af8009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30882863b40917f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.162+0000 7f1af6000700 1 -- 172.21.15.16:0/2746502280 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8004230 con 0x563ef0306820 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.163+0000 7f1af6000700 1 -- 172.21.15.16:0/2746502280 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af80043d0 con 0x563ef0306820 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.163+0000 7f1af6000700 1 -- 172.21.15.16:0/2746502280 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8004e50 con 0x563ef0306820 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 msgr2=0x563ef0313020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 0x563ef0313020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009bd0 tx=0x7f1af8009ee0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 shutdown_connections 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0306820 0x563ef0313020 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0187e40 0x563ef0305ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/2746502280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.164+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 >> 172.21.15.16:0/2746502280 conn(0x563ef01e7fc0 msgr2=0x563ef0301d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.166+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 shutdown_connections 2024-05-27T07:07:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.166+0000 7f1b160d1b80 1 -- 172.21.15.16:0/2746502280 wait complete. 2024-05-27T07:07:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.167+0000 7f1b160d1b80 1 -- start start 2024-05-27T07:07:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.168+0000 7f1b160d1b80 1 Processor -- start 2024-05-27T07:07:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.170+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.170+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.170+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 0x563ef03110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.171+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ef012afa0 con 0x563ef0306820 2024-05-27T07:07:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.171+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ef01c5760 con 0x563ef0187e40 2024-05-27T07:07:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.171+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ef0313990 con 0x563ef0306410 2024-05-27T07:07:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.171+0000 7f1b160d1b80 10 client.0 _ll_register_callbacks cb 0x563ef01cec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.172+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.172+0000 7f1affe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.172+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60418/0 (socket says 172.21.15.16:60418) 2024-05-27T07:07:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.173+0000 7f1b04c00700 1 -- 172.21.15.16:0/3290918636 learned_addr learned my addr 172.21.15.16:0/3290918636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.172+0000 7f1affe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60104/0 (socket says 172.21.15.16:60104) 2024-05-27T07:07:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.174+0000 7f1b05600700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 0x563ef03110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b05600700 1 -- 172.21.15.16:0/3290918636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 msgr2=0x563ef0310b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b05600700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b05600700 1 -- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 msgr2=0x563ef0310630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b05600700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b05600700 1 -- 172.21.15.16:0/3290918636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef01748d0 con 0x563ef0306820 2024-05-27T07:07:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.175+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:07:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.177+0000 7f1aeaa00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:07:53.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.177+0000 7f1b05600700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 0x563ef03110b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009bd0 tx=0x7f1af800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639d8833e83d67cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.178+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8005230 con 0x563ef0306820 2024-05-27T07:07:53.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.179+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1af80053d0 con 0x563ef0306820 2024-05-27T07:07:53.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.181+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af80043d0 con 0x563ef0306820 2024-05-27T07:07:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.181+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1af80055d0 con 0x563ef0306820 2024-05-27T07:07:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.182+0000 7f1b160d1b80 10 client.16230 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:53.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.182+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ef01f5060 con 0x563ef0306820 2024-05-27T07:07:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af80043d0 con 0x563ef0306820 2024-05-27T07:07:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1af8010430 con 0x563ef0306820 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 10 client.16230 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 20 client.16230 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef03208c0 0x563ef0322ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 4 client.16230 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:07:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.185+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef0236d90 con 0x563ef03208c0 2024-05-27T07:07:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.186+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef03208c0 0x563ef0322ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.187+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef03208c0 0x563ef0322ce0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb613ecbed41585d server_cookie=e9270a40df8bfb15 in_seq=0 out_seq=0 2024-05-27T07:07:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.187+0000 7f1aeaa00700 10 client.16230 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:07:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.189+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3290918636 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ef0236d90 con 0x563ef03208c0 2024-05-27T07:07:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.189+0000 7f1aeaa00700 10 client.16230 handle_command_reply: tid=0 2024-05-27T07:07:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.189+0000 7f1b160d1b80 1 client.16230 shutdown 2024-05-27T07:07:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 msgr2=0x563ef03110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 0x563ef03110b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009bd0 tx=0x7f1af800eb60 comp rx=0 tx=0).stop 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 shutdown_connections 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef03208c0 0x563ef0322ce0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306820 0x563ef03110b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef0310630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3290918636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef0310b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 >> 172.21.15.16:0/3290918636 conn(0x563ef01e7fc0 msgr2=0x563ef0300ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1aeaa00700 0 client.16230 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:07:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.190+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 shutdown_connections 2024-05-27T07:07:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.191+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3290918636 wait complete. 2024-05-27T07:07:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.192+0000 7f1b160d1b80 20 client.16230 trim_cache size 0 max 16384 2024-05-27T07:07:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.228+0000 7f1b160d1b80 1 Processor -- start 2024-05-27T07:07:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 -- start start 2024-05-27T07:07:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef030a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef031ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c7960 con 0x563ef01e7fc0 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c94a0 con 0x563ef0187e40 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.229+0000 7f1b160d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c8030 con 0x563ef0306410 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.231+0000 7f1affe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef030a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.231+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.231+0000 7f1b05600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef031ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.231+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59516/0 (socket says 172.21.15.16:59516) 2024-05-27T07:07:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.231+0000 7f1b04c00700 1 -- 172.21.15.16:0/4162552264 learned_addr learned my addr 172.21.15.16:0/4162552264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.232+0000 7f1b04c00700 1 -- 172.21.15.16:0/4162552264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 msgr2=0x563ef031ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.232+0000 7f1b04c00700 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef031ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.232+0000 7f1b04c00700 1 -- 172.21.15.16:0/4162552264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 msgr2=0x563ef030a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.232+0000 7f1b04c00700 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef030a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.232+0000 7f1b04c00700 1 -- 172.21.15.16:0/4162552264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef0303350 con 0x563ef01e7fc0 2024-05-27T07:07:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.234+0000 7f1b04c00700 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cd90 tx=0x7f1af000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555d811086141e6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.234+0000 7f1af5600700 1 -- 172.21.15.16:0/4162552264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af001cc70 con 0x563ef01e7fc0 2024-05-27T07:07:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.234+0000 7f1af5600700 1 -- 172.21.15.16:0/4162552264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af0027030 con 0x563ef01e7fc0 2024-05-27T07:07:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.234+0000 7f1af5600700 1 -- 172.21.15.16:0/4162552264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af001cc70 con 0x563ef01e7fc0 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.236+0000 7f1b160d1b80 1 -- 172.21.15.16:0/4162552264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 msgr2=0x563ef030be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.236+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cd90 tx=0x7f1af000f000 comp rx=0 tx=0).stop 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.237+0000 7f1b160d1b80 1 -- 172.21.15.16:0/4162552264 shutdown_connections 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.237+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef0306410 0x563ef031ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.237+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef01e7fc0 0x563ef030be30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.237+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/4162552264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef030a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.237+0000 7f1b160d1b80 1 -- 172.21.15.16:0/4162552264 >> 172.21.15.16:0/4162552264 conn(0x563ef02faf60 msgr2=0x563ef030ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.238+0000 7f1b160d1b80 1 -- 172.21.15.16:0/4162552264 shutdown_connections 2024-05-27T07:07:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.238+0000 7f1b160d1b80 1 -- 172.21.15.16:0/4162552264 wait complete. 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.239+0000 7f1b160d1b80 1 -- start start 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.240+0000 7f1b160d1b80 1 Processor -- start 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.241+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.242+0000 7f1b160d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 0x563ef0314720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.242+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.242+0000 7f1b04c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60434/0 (socket says 172.21.15.16:60434) 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.242+0000 7f1b04c00700 1 -- 172.21.15.16:0/3277320213 learned_addr learned my addr 172.21.15.16:0/3277320213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.243+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 0x563ef03891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.243+0000 7f1affe00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 0x563ef0314720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.244+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c8dd0 con 0x563ef0306410 2024-05-27T07:07:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.244+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c8700 con 0x563ef0187e40 2024-05-27T07:07:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.244+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001c5900 con 0x563ef01e7fc0 2024-05-27T07:07:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.246+0000 7f1b05600700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 0x563ef03891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.246+0000 7f1b160d1b80 10 client.0 _ll_register_callbacks cb 0x563ef01cec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:07:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1affe00700 1 -- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 msgr2=0x563ef03141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1affe00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1affe00700 1 -- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 msgr2=0x563ef03891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:07:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1affe00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 0x563ef03891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1affe00700 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef0313520 con 0x563ef01e7fc0 2024-05-27T07:07:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:07:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.247+0000 7f1b05600700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 0x563ef03891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:07:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.251+0000 7f1aeaa00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:07:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.251+0000 7f1affe00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 0x563ef0314720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0e0640 tx=0x7f1aec0aa6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab3357d4c2ec6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.252+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0a9d70 con 0x563ef01e7fc0 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.252+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1aec0ae370 con 0x563ef01e7fc0 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.252+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af0027030 con 0x563ef01e7fc0 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.253+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1aec0aa960 con 0x563ef01e7fc0 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.253+0000 7f1b160d1b80 10 client.25999 fetch_fsmap learned FSMap version 31 2024-05-27T07:07:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.253+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ef038a740 con 0x563ef01e7fc0 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.253+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af0027030 con 0x563ef01e7fc0 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.253+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1aec012070 con 0x563ef01e7fc0 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 20 client.25999 populate_metadata read hostname 'smithi016' 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef0306820 0x563ef038ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 4 client.25999 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:07:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.254+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ef0230570 con 0x563ef0306820 2024-05-27T07:07:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.255+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef0306820 0x563ef038ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:07:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.256+0000 7f1b04c00700 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef0306820 0x563ef038ce10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cfc981e4303820 server_cookie=5fc3aae4cf04816 in_seq=0 out_seq=0 2024-05-27T07:07:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.256+0000 7f1aeaa00700 10 client.25999 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:07:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.256+0000 7f1aeaa00700 1 -- 172.21.15.16:0/3277320213 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ef0230570 con 0x563ef0306820 2024-05-27T07:07:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.256+0000 7f1aeaa00700 10 client.25999 handle_command_reply: tid=0 2024-05-27T07:07:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.257+0000 7f1b160d1b80 1 client.25999 shutdown 2024-05-27T07:07:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.257+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 msgr2=0x563ef0314720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.257+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 0x563ef0314720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0e0640 tx=0x7f1aec0aa6f0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 shutdown_connections 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x563ef0306820 0x563ef038ce10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ef0306410 0x563ef03891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ef0187e40 0x563ef03141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 --2- 172.21.15.16:0/3277320213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ef01e7fc0 0x563ef0314720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 >> 172.21.15.16:0/3277320213 conn(0x563ef02faf60 msgr2=0x563ef0310410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1aeaa00700 0 client.25999 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:07:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.258+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 shutdown_connections 2024-05-27T07:07:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.259+0000 7f1b160d1b80 1 -- 172.21.15.16:0/3277320213 wait complete. 2024-05-27T07:07:53.252 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:07:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.259+0000 7f1b160d1b80 20 client.25999 trim_cache size 0 max 16384 2024-05-27T07:07:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.262+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 msgr2=0x7f1ae40b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 0x7f1ae40b4f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1af000ab90 tx=0x7f1af0009250 comp rx=0 tx=0).stop 2024-05-27T07:07:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 msgr2=0x7f1b0011d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005b40 tx=0x7f1aec003d10 comp rx=0 tx=0).stop 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 shutdown_connections 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f1ae40b2a60 0x7f1ae40b4f70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b00110ba0 0x7f1b00124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b0010ac00 0x7f1b0011d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 --2- 172.21.15.16:0/4154700395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b0010b550 0x7f1b0011d730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:07:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 >> 172.21.15.16:0/4154700395 conn(0x7f1b0006f280 msgr2=0x7f1b000726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:07:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.263+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 shutdown_connections 2024-05-27T07:07:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:07:53.264+0000 7f1b06000700 1 -- 172.21.15.16:0/4154700395 wait complete. 2024-05-27T07:07:54.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:53 smithi164 ceph-mon[35327]: pgmap v212: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 39 MiB/s wr, 641 op/s 2024-05-27T07:07:54.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:53 smithi016 ceph-mon[31415]: pgmap v212: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 39 MiB/s wr, 641 op/s 2024-05-27T07:07:54.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:53 smithi069 ceph-mon[35276]: pgmap v212: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 39 MiB/s wr, 641 op/s 2024-05-27T07:07:55.464 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:07:55.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:07:55.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:55 smithi016 ceph-mon[31415]: pgmap v213: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:56.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:55 smithi164 ceph-mon[35327]: pgmap v213: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:56.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:55 smithi069 ceph-mon[35276]: pgmap v213: 129 pgs: 129 active+clean; 16 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 31 MiB/s wr, 497 op/s 2024-05-27T07:07:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:07:58.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:57 smithi164 ceph-mon[35327]: pgmap v214: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 41 MiB/s wr, 674 op/s 2024-05-27T07:07:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:57 smithi016 ceph-mon[31415]: pgmap v214: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 41 MiB/s wr, 674 op/s 2024-05-27T07:07:58.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:57 smithi069 ceph-mon[35276]: pgmap v214: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 41 MiB/s wr, 674 op/s 2024-05-27T07:08:00.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:07:59 smithi164 ceph-mon[35327]: pgmap v215: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 29 MiB/s wr, 474 op/s 2024-05-27T07:08:00.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:07:59 smithi016 ceph-mon[31415]: pgmap v215: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 29 MiB/s wr, 474 op/s 2024-05-27T07:08:00.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:07:59 smithi069 ceph-mon[35276]: pgmap v215: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 29 MiB/s wr, 474 op/s 2024-05-27T07:08:02.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:01 smithi164 ceph-mon[35327]: pgmap v216: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 MiB/s wr, 507 op/s 2024-05-27T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:01 smithi016 ceph-mon[31415]: pgmap v216: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 MiB/s wr, 507 op/s 2024-05-27T07:08:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:01 smithi069 ceph-mon[35276]: pgmap v216: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 MiB/s wr, 507 op/s 2024-05-27T07:08:04.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:03 smithi164 ceph-mon[35327]: pgmap v217: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 42 MiB/s wr, 682 op/s 2024-05-27T07:08:04.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:03 smithi016 ceph-mon[31415]: pgmap v217: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 42 MiB/s wr, 682 op/s 2024-05-27T07:08:04.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:03 smithi069 ceph-mon[35276]: pgmap v217: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 42 MiB/s wr, 682 op/s 2024-05-27T07:08:06.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:05 smithi164 ceph-mon[35327]: pgmap v218: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:08:06.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:05 smithi016 ceph-mon[31415]: pgmap v218: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:08:06.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:05 smithi069 ceph-mon[35276]: pgmap v218: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 29 MiB/s wr, 477 op/s 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.599+0000 7fb88d400700 1 -- 172.21.15.16:0/2664776419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888075b50 msgr2=0x7fb888075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.599+0000 7fb88d400700 1 --2- 172.21.15.16:0/2664776419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888075b50 0x7fb888075f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb878009990 tx=0x7fb878009ca0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.600+0000 7fb88d400700 1 -- 172.21.15.16:0/2664776419 shutdown_connections 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.600+0000 7fb88d400700 1 --2- 172.21.15.16:0/2664776419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb888115170 0x7fb888117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.600+0000 7fb88d400700 1 --2- 172.21.15.16:0/2664776419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888075b50 0x7fb888075f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.600+0000 7fb88d400700 1 --2- 172.21.15.16:0/2664776419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8880764a0 0x7fb888114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.600+0000 7fb88d400700 1 -- 172.21.15.16:0/2664776419 >> 172.21.15.16:0/2664776419 conn(0x7fb888079d00 msgr2=0x7fb88807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.601+0000 7fb88d400700 1 -- 172.21.15.16:0/2664776419 shutdown_connections 2024-05-27T07:08:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.601+0000 7fb88d400700 1 -- 172.21.15.16:0/2664776419 wait complete. 2024-05-27T07:08:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.604+0000 7fb88d400700 1 Processor -- start 2024-05-27T07:08:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.604+0000 7fb88d400700 1 -- start start 2024-05-27T07:08:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.606+0000 7fb88d400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb888075b50 0x7fb888114bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.607+0000 7fb88d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.607+0000 7fb88d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888115170 0x7fb88810e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.607+0000 7fb88d400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb88811a780 con 0x7fb888075b50 2024-05-27T07:08:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.607+0000 7fb88d400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb88811a600 con 0x7fb8880764a0 2024-05-27T07:08:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.607+0000 7fb88d400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb88811a900 con 0x7fb888115170 2024-05-27T07:08:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.608+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.608+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35738/0 (socket says 172.21.15.16:35738) 2024-05-27T07:08:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.608+0000 7fb87fe00700 1 -- 172.21.15.16:0/2314444821 learned_addr learned my addr 172.21.15.16:0/2314444821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb887200700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888115170 0x7fb88810e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb886800700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb888075b50 0x7fb888114bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb87fe00700 1 -- 172.21.15.16:0/2314444821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888115170 msgr2=0x7fb88810e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb87fe00700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888115170 0x7fb88810e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb87fe00700 1 -- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb888075b50 msgr2=0x7fb888114bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb87fe00700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb888075b50 0x7fb888114bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.609+0000 7fb87fe00700 1 -- 172.21.15.16:0/2314444821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb888112720 con 0x7fb8880764a0 2024-05-27T07:08:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.610+0000 7fb87fe00700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9a0 tx=0x7fb87000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.611+0000 7fb87f400700 1 -- 172.21.15.16:0/2314444821 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87000cbd0 con 0x7fb8880764a0 2024-05-27T07:08:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.611+0000 7fb87f400700 1 -- 172.21.15.16:0/2314444821 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb87000cd70 con 0x7fb8880764a0 2024-05-27T07:08:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.611+0000 7fb87f400700 1 -- 172.21.15.16:0/2314444821 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870018680 con 0x7fb8880764a0 2024-05-27T07:08:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.611+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb888112a10 con 0x7fb8880764a0 2024-05-27T07:08:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.612+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb888112f00 con 0x7fb8880764a0 2024-05-27T07:08:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.619+0000 7fb87f400700 1 -- 172.21.15.16:0/2314444821 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb870010430 con 0x7fb8880764a0 2024-05-27T07:08:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.621+0000 7fb87f400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 0x7fb8680b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.621+0000 7fb87f400700 1 -- 172.21.15.16:0/2314444821 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb8700e2b50 con 0x7fb8880764a0 2024-05-27T07:08:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.625+0000 7fb886800700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 0x7fb8680b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.626+0000 7fb886800700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 0x7fb8680b50d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b2e0 tx=0x7fb878005fb0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.659+0000 7fb89bff2b80 1 Processor -- start 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 -- start start 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99ee95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99eea0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b99edc4ec0 con 0x55b99ee96490 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b99edb4a60 con 0x55b99ed17ec0 2024-05-27T07:08:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.660+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b99edb8aa0 con 0x55b99ee968a0 2024-05-27T07:08:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.661+0000 7fb887200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.661+0000 7fb887200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33888/0 (socket says 172.21.15.16:33888) 2024-05-27T07:08:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.661+0000 7fb887200700 1 -- 172.21.15.16:0/216257679 learned_addr learned my addr 172.21.15.16:0/216257679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.661+0000 7fb87fe00700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99eea0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.661+0000 7fb886800700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99ee95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb887200700 1 -- 172.21.15.16:0/216257679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 msgr2=0x55b99ee95f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb887200700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99ee95f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb887200700 1 -- 172.21.15.16:0/216257679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 msgr2=0x55b99eea0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb887200700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99eea0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb887200700 1 -- 172.21.15.16:0/216257679 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99ed7d4a0 con 0x55b99ee968a0 2024-05-27T07:08:06.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb87fe00700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99eea0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:08:06.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.662+0000 7fb886800700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99ee95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:08:06.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.663+0000 7fb887200700 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb880003fa0 tx=0x7fb880009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbc43bb2516f304 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.663+0000 7fb877e00700 1 -- 172.21.15.16:0/216257679 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880004bb0 con 0x55b99ee968a0 2024-05-27T07:08:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.664+0000 7fb877e00700 1 -- 172.21.15.16:0/216257679 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb880004d50 con 0x55b99ee968a0 2024-05-27T07:08:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.664+0000 7fb877e00700 1 -- 172.21.15.16:0/216257679 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880007d90 con 0x55b99ee968a0 2024-05-27T07:08:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.666+0000 7fb89bff2b80 1 -- 172.21.15.16:0/216257679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 msgr2=0x55b99eea30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.666+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb880003fa0 tx=0x7fb880009ba0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.667+0000 7fb89bff2b80 1 -- 172.21.15.16:0/216257679 shutdown_connections 2024-05-27T07:08:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.667+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee968a0 0x55b99eea30a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.667+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99ee95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.668+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/216257679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99eea0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.668+0000 7fb89bff2b80 1 -- 172.21.15.16:0/216257679 >> 172.21.15.16:0/216257679 conn(0x55b99ed78040 msgr2=0x55b99ee91db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.668+0000 7fb89bff2b80 1 -- 172.21.15.16:0/216257679 shutdown_connections 2024-05-27T07:08:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.669+0000 7fb89bff2b80 1 -- 172.21.15.16:0/216257679 wait complete. 2024-05-27T07:08:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.670+0000 7fb89bff2b80 1 -- start start 2024-05-27T07:08:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.671+0000 7fb89bff2b80 1 Processor -- start 2024-05-27T07:08:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.673+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99eea06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.674+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 0x55b99eea0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.674+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.674+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b99ecbb020 con 0x55b99ee968a0 2024-05-27T07:08:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.674+0000 7fb886800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99eea06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b99ed557e0 con 0x55b99ed17ec0 2024-05-27T07:08:06.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b99eea3a10 con 0x55b99ee96490 2024-05-27T07:08:06.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb89bff2b80 10 client.0 _ll_register_callbacks cb 0x55b99ed5ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:06.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb886800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99eea06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35742/0 (socket says 172.21.15.16:35742) 2024-05-27T07:08:06.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb886800700 1 -- 172.21.15.16:0/1898086156 learned_addr learned my addr 172.21.15.16:0/1898086156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:06.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.675+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 0x55b99eea0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.676+0000 7fb887200700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.677+0000 7fb87fe00700 1 -- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 msgr2=0x55b99eea06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.677+0000 7fb87fe00700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99eea06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.677+0000 7fb87fe00700 1 -- 172.21.15.16:0/1898086156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 msgr2=0x55b99eea1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.677+0000 7fb87fe00700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.677+0000 7fb87fe00700 1 -- 172.21.15.16:0/1898086156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99ed04950 con 0x55b99ee96490 2024-05-27T07:08:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.678+0000 7fb874c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.678+0000 7fb87fe00700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 0x55b99eea0c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8700dfeb0 tx=0x7fb8700a9f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7749742a11aed4be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.679+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b2040 con 0x55b99ee96490 2024-05-27T07:08:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.679+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb8700a6030 con 0x55b99ee96490 2024-05-27T07:08:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.680+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb880004d50 con 0x55b99ee96490 2024-05-27T07:08:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.681+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb8700b2040 con 0x55b99ee96490 2024-05-27T07:08:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.681+0000 7fb89bff2b80 10 client.26011 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.681+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b99ed850e0 con 0x55b99ee96490 2024-05-27T07:08:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.682+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8700a9550 con 0x55b99ee96490 2024-05-27T07:08:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.682+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb8700cc0e0 con 0x55b99ee96490 2024-05-27T07:08:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 10 client.26011 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:08:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 20 client.26011 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99eeb09f0 0x55b99eeb2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 4 client.26011 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:08:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.683+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb880004d50 con 0x55b99eeb09f0 2024-05-27T07:08:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.684+0000 7fb886800700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99eeb09f0 0x55b99eeb2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.687+0000 7fb886800700 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99eeb09f0 0x55b99eeb2e10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc8486cb8f5271f server_cookie=709f5c699a5c6ec3 in_seq=0 out_seq=0 2024-05-27T07:08:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.687+0000 7fb874c00700 10 client.26011 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.688+0000 7fb874c00700 1 -- 172.21.15.16:0/1898086156 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fb880004d50 con 0x55b99eeb09f0 2024-05-27T07:08:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.688+0000 7fb874c00700 10 client.26011 handle_command_reply: tid=0 2024-05-27T07:08:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.689+0000 7fb89bff2b80 1 client.26011 shutdown 2024-05-27T07:08:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 msgr2=0x55b99eea0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 0x55b99eea0c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8700dfeb0 tx=0x7fb8700a9f80 comp rx=0 tx=0).stop 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 shutdown_connections 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99eeb09f0 0x55b99eeb2e10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb874c00700 0 client.26011 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ed17ec0 0x55b99eea06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/1898086156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ee96490 0x55b99eea0c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.690+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 >> 172.21.15.16:0/1898086156 conn(0x55b99ed78040 msgr2=0x55b99ee91070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.691+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 shutdown_connections 2024-05-27T07:08:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.691+0000 7fb89bff2b80 1 -- 172.21.15.16:0/1898086156 wait complete. 2024-05-27T07:08:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.691+0000 7fb89bff2b80 20 client.26011 trim_cache size 0 max 16384 2024-05-27T07:08:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.744+0000 7fb89bff2b80 1 Processor -- start 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.744+0000 7fb89bff2b80 1 -- start start 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.744+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 0x55b99eea2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.744+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99ee9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.745+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 0x55b99eea0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.745+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881bf270 con 0x55b99ee96490 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.745+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881c0db0 con 0x55b99ee968a0 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.745+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881bf940 con 0x55b99ed17ec0 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.746+0000 7fb887200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 0x55b99eea0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.746+0000 7fb887200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 0x55b99eea0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35754/0 (socket says 172.21.15.16:35754) 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.746+0000 7fb887200700 1 -- 172.21.15.16:0/2649046631 learned_addr learned my addr 172.21.15.16:0/2649046631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.746+0000 7fb87fe00700 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99ee9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.746+0000 7fb886800700 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 0x55b99eea2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 -- 172.21.15.16:0/2649046631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 msgr2=0x55b99eea0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 0x55b99eea0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 -- 172.21.15.16:0/2649046631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 msgr2=0x55b99ee9beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99ee9beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 -- 172.21.15.16:0/2649046631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99ee8c250 con 0x55b99ed17ec0 2024-05-27T07:08:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.747+0000 7fb886800700 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 0x55b99eea2b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b99eeb3500 tx=0x7fb87801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd05c653d8619db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.748+0000 7fb877400700 1 -- 172.21.15.16:0/2649046631 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878035070 con 0x55b99ed17ec0 2024-05-27T07:08:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.748+0000 7fb877400700 1 -- 172.21.15.16:0/2649046631 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb878003b10 con 0x55b99ed17ec0 2024-05-27T07:08:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.748+0000 7fb877400700 1 -- 172.21.15.16:0/2649046631 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8780309a0 con 0x55b99ed17ec0 2024-05-27T07:08:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 -- 172.21.15.16:0/2649046631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 msgr2=0x55b99eea2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 0x55b99eea2b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b99eeb3500 tx=0x7fb87801d000 comp rx=0 tx=0).stop 2024-05-27T07:08:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 -- 172.21.15.16:0/2649046631 shutdown_connections 2024-05-27T07:08:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee968a0 0x55b99eea0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99ed17ec0 0x55b99eea2b80 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b99eeb3500 tx=0x7fb87801d000 comp rx=0 tx=0).stop 2024-05-27T07:08:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/2649046631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee96490 0x55b99ee9beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.750+0000 7fb89bff2b80 1 -- 172.21.15.16:0/2649046631 >> 172.21.15.16:0/2649046631 conn(0x55b99ed78040 msgr2=0x55b99eea04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.752+0000 7fb89bff2b80 1 -- 172.21.15.16:0/2649046631 shutdown_connections 2024-05-27T07:08:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.752+0000 7fb89bff2b80 1 -- 172.21.15.16:0/2649046631 wait complete. 2024-05-27T07:08:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.754+0000 7fb89bff2b80 1 -- start start 2024-05-27T07:08:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.754+0000 7fb89bff2b80 1 Processor -- start 2024-05-27T07:08:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 0x55b99eea93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 0x55b99eea60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881c06e0 con 0x55b99ee968a0 2024-05-27T07:08:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881c0010 con 0x55b99ee96490 2024-05-27T07:08:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.755+0000 7fb89bff2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881bd210 con 0x55b99eea5c90 2024-05-27T07:08:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb89bff2b80 10 client.0 _ll_register_callbacks cb 0x55b99ee92370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb886800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 0x55b99eea93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb886800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 0x55b99eea93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35768/0 (socket says 172.21.15.16:35768) 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb886800700 1 -- 172.21.15.16:0/3468640610 learned_addr learned my addr 172.21.15.16:0/3468640610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb887200700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 0x55b99eea60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.756+0000 7fb87fe00700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.757+0000 7fb887200700 1 -- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 msgr2=0x55b99eea93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.757+0000 7fb887200700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 0x55b99eea93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.757+0000 7fb887200700 1 -- 172.21.15.16:0/3468640610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 msgr2=0x55b99eea5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.757+0000 7fb887200700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.757+0000 7fb887200700 1 -- 172.21.15.16:0/3468640610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99ee933d0 con 0x55b99eea5c90 2024-05-27T07:08:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.758+0000 7fb887200700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 0x55b99eea60c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b99ee916a0 tx=0x7fb880007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787b10c64d2a30b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.758+0000 7fb874c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.760+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880007ae0 con 0x55b99eea5c90 2024-05-27T07:08:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.760+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb880007c80 con 0x55b99eea5c90 2024-05-27T07:08:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.760+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb878003b10 con 0x55b99eea5c90 2024-05-27T07:08:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.761+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb880015480 con 0x55b99eea5c90 2024-05-27T07:08:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.761+0000 7fb89bff2b80 10 client.26017 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.761+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b99ef1c640 con 0x55b99eea5c90 2024-05-27T07:08:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.762+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb878003b10 con 0x55b99eea5c90 2024-05-27T07:08:06.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.764+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb880031420 con 0x55b99eea5c90 2024-05-27T07:08:06.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.764+0000 7fb89bff2b80 10 client.26017 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:06.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb89bff2b80 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:08:06.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb89bff2b80 20 client.26017 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:06.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99ef1c870 0x55b99ef1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb89bff2b80 4 client.26017 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:08:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b99edc6e10 con 0x55b99ef1c870 2024-05-27T07:08:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.765+0000 7fb886800700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99ef1c870 0x55b99ef1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.767+0000 7fb886800700 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99ef1c870 0x55b99ef1ec90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b71f5025f51ca16 server_cookie=ef7b08c1af0f5437 in_seq=0 out_seq=0 2024-05-27T07:08:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.767+0000 7fb874c00700 10 client.26017 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.768+0000 7fb874c00700 1 -- 172.21.15.16:0/3468640610 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b99edc6e10 con 0x55b99ef1c870 2024-05-27T07:08:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.768+0000 7fb874c00700 10 client.26017 handle_command_reply: tid=0 2024-05-27T07:08:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.768+0000 7fb89bff2b80 1 client.26017 shutdown 2024-05-27T07:08:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.770+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 msgr2=0x55b99eea60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.770+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 0x55b99eea60c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b99ee916a0 tx=0x7fb880007730 comp rx=0 tx=0).stop 2024-05-27T07:08:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 shutdown_connections 2024-05-27T07:08:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55b99ef1c870 0x55b99ef1ec90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b99eea5c90 0x55b99eea60c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b99ee96490 0x55b99eea93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb874c00700 0 client.26017 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 --2- 172.21.15.16:0/3468640610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b99ee968a0 0x55b99eea5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.773+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 >> 172.21.15.16:0/3468640610 conn(0x55b99ed78040 msgr2=0x55b99eea04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.774+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 shutdown_connections 2024-05-27T07:08:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.774+0000 7fb89bff2b80 1 -- 172.21.15.16:0/3468640610 wait complete. 2024-05-27T07:08:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.774+0000 7fb89bff2b80 20 client.26017 trim_cache size 0 max 16384 2024-05-27T07:08:06.766 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:08:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:08:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ee2a9b4a-f933-4701-ad94-c219bb9e3c38", 2024-05-27T07:08:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:08:06.766 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 msgr2=0x7fb8680b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 0x7fb8680b50d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b2e0 tx=0x7fb878005fb0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 msgr2=0x7fb88810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9a0 tx=0x7fb87000ecb0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 shutdown_connections 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fb8680b2bc0 0x7fb8680b50d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb888115170 0x7fb88810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb888075b50 0x7fb888114bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 --2- 172.21.15.16:0/2314444821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8880764a0 0x7fb88810dd20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.776+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 >> 172.21.15.16:0/2314444821 conn(0x7fb888079d00 msgr2=0x7fb888101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.777+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 shutdown_connections 2024-05-27T07:08:06.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:06.778+0000 7fb88d400700 1 -- 172.21.15.16:0/2314444821 wait complete. 2024-05-27T07:08:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:07 smithi164 ceph-mon[35327]: pgmap v219: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 41 MiB/s wr, 695 op/s 2024-05-27T07:08:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:07 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:08:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:07 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:08.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:07 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:08:08.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:07 smithi016 ceph-mon[31415]: pgmap v219: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 41 MiB/s wr, 695 op/s 2024-05-27T07:08:08.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:07 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:08:08.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:07 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:08.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:07 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:08:08.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:07 smithi069 ceph-mon[35276]: pgmap v219: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 41 MiB/s wr, 695 op/s 2024-05-27T07:08:08.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:07 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:08:08.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:07 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:08.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:07 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:08:09.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:08 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:09.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:08 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:09.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:08 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:08:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:09 smithi164 ceph-mon[35327]: pgmap v220: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 29 MiB/s wr, 486 op/s 2024-05-27T07:08:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:09 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:08:10.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:09 smithi164 ceph-mon[35327]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:08:10.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:09 smithi016 ceph-mon[31415]: pgmap v220: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 29 MiB/s wr, 486 op/s 2024-05-27T07:08:10.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:09 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:08:10.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:09 smithi016 ceph-mon[31415]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:08:10.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:09 smithi069 ceph-mon[35276]: pgmap v220: 129 pgs: 129 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 29 MiB/s wr, 486 op/s 2024-05-27T07:08:10.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:09 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:08:10.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:09 smithi069 ceph-mon[35276]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:08:11.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:08:11.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:11 smithi016 ceph-mon[31415]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 34 MiB/s wr, 575 op/s 2024-05-27T07:08:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:08:12.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:11 smithi069 ceph-mon[35276]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 34 MiB/s wr, 575 op/s 2024-05-27T07:08:12.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:11 smithi164 ceph-mon[35327]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 34 MiB/s wr, 575 op/s 2024-05-27T07:08:13.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:12 smithi016 ceph-mon[31415]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 32 MiB/s wr, 549 op/s 2024-05-27T07:08:13.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:12 smithi069 ceph-mon[35276]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 32 MiB/s wr, 549 op/s 2024-05-27T07:08:13.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:12 smithi164 ceph-mon[35327]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 32 MiB/s wr, 549 op/s 2024-05-27T07:08:14.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:14 smithi069 ceph-mon[35276]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:08:15.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:14 smithi164 ceph-mon[35327]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:08:15.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:14 smithi016 ceph-mon[31415]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:08:16.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:15 smithi164 ceph-mon[35327]: pgmap v225: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 312 op/s 2024-05-27T07:08:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:15 smithi016 ceph-mon[31415]: pgmap v225: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 312 op/s 2024-05-27T07:08:16.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:15 smithi069 ceph-mon[35276]: pgmap v225: 129 pgs: 2 active+clean+snaptrim, 124 active+clean, 3 active+clean+snaptrim_wait; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 312 op/s 2024-05-27T07:08:18.041 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:17 smithi016 ceph-mon[31415]: pgmap v226: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-05-27T07:08:18.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:17 smithi164 ceph-mon[35327]: pgmap v226: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-05-27T07:08:18.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:17 smithi069 ceph-mon[35276]: pgmap v226: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-05-27T07:08:19.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:19 smithi016 ceph-mon[31415]: pgmap v227: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 34 MiB/s wr, 571 op/s 2024-05-27T07:08:19.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:19 smithi016 ceph-mon[31415]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:08:20.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:19 smithi164 ceph-mon[35327]: pgmap v227: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 34 MiB/s wr, 571 op/s 2024-05-27T07:08:20.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:19 smithi164 ceph-mon[35327]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:08:20.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:19 smithi069 ceph-mon[35276]: pgmap v227: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 34 MiB/s wr, 571 op/s 2024-05-27T07:08:20.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:19 smithi069 ceph-mon[35276]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:08:21.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 -- 172.21.15.16:0/2973488472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 msgr2=0x7fbfcc112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 --2- 172.21.15.16:0/2973488472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc112ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009f90 tx=0x7fbfc400b330 comp rx=0 tx=0).stop 2024-05-27T07:08:21.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 -- 172.21.15.16:0/2973488472 shutdown_connections 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 --2- 172.21.15.16:0/2973488472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc112ff0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 --2- 172.21.15.16:0/2973488472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc075d20 0x7fbfcc076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 --2- 172.21.15.16:0/2973488472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc076670 0x7fbfcc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.791+0000 7fbfd3200700 1 -- 172.21.15.16:0/2973488472 >> 172.21.15.16:0/2973488472 conn(0x7fbfcc100030 msgr2=0x7fbfcc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.793+0000 7fbfd3200700 1 -- 172.21.15.16:0/2973488472 shutdown_connections 2024-05-27T07:08:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.793+0000 7fbfd3200700 1 -- 172.21.15.16:0/2973488472 wait complete. 2024-05-27T07:08:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.795+0000 7fbfd3200700 1 Processor -- start 2024-05-27T07:08:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.796+0000 7fbfd3200700 1 -- start start 2024-05-27T07:08:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.797+0000 7fbfd3200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc075d20 0x7fbfcc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.797+0000 7fbfd3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc076670 0x7fbfcc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.797+0000 7fbfd3200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd3200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc116570 con 0x7fbfcc075d20 2024-05-27T07:08:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd3200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1163f0 con 0x7fbfcc110b90 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1166f0 con 0x7fbfcc076670 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34876/0 (socket says 172.21.15.16:34876) 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.799+0000 7fbfd1200700 1 -- 172.21.15.16:0/3997569095 learned_addr learned my addr 172.21.15.16:0/3997569095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfcb400700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc076670 0x7fbfcc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfd1200700 1 -- 172.21.15.16:0/3997569095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc076670 msgr2=0x7fbfcc19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfd1200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc076670 0x7fbfcc19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfd1200700 1 -- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc075d20 msgr2=0x7fbfcc19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfd1200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc075d20 0x7fbfcc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.800+0000 7fbfd1200700 1 -- 172.21.15.16:0/3997569095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc1a1df0 con 0x7fbfcc110b90 2024-05-27T07:08:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.802+0000 7fbfd1200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009f60 tx=0x7fbfc400ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.802+0000 7fbfc8c00700 1 -- 172.21.15.16:0/3997569095 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4021070 con 0x7fbfcc110b90 2024-05-27T07:08:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.802+0000 7fbfc8c00700 1 -- 172.21.15.16:0/3997569095 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc4007ac0 con 0x7fbfcc110b90 2024-05-27T07:08:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.802+0000 7fbfc8c00700 1 -- 172.21.15.16:0/3997569095 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40052a0 con 0x7fbfcc110b90 2024-05-27T07:08:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.803+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc1a2080 con 0x7fbfcc110b90 2024-05-27T07:08:21.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.803+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1a2570 con 0x7fbfcc110b90 2024-05-27T07:08:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.820+0000 7fbfc8c00700 1 -- 172.21.15.16:0/3997569095 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbfc4024070 con 0x7fbfcc110b90 2024-05-27T07:08:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.822+0000 7fbfc8c00700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 0x7fbfb00b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.823+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 0x7fbfb00b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.825+0000 7fbfc8c00700 1 -- 172.21.15.16:0/3997569095 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7fbfc400f020 con 0x7fbfcc110b90 2024-05-27T07:08:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.827+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 0x7fbfb00b50d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005e50 tx=0x7fbfb4005da0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.840+0000 7fbfe1da3b80 1 Processor -- start 2024-05-27T07:08:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.843+0000 7fbfe1da3b80 1 -- start start 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a7f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609919ace40 con 0x560991a7edb0 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56099199c9e0 con 0x5609918ffe40 2024-05-27T07:08:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.844+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5609919a0a20 con 0x560991a84d30 2024-05-27T07:08:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.845+0000 7fbfcbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.845+0000 7fbfcbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a847f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:34880/0 (socket says 172.21.15.16:34880) 2024-05-27T07:08:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.845+0000 7fbfcbe00700 1 -- 172.21.15.16:0/2004493737 learned_addr learned my addr 172.21.15.16:0/2004493737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.845+0000 7fbfcb400700 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a7e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 -- 172.21.15.16:0/2004493737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 msgr2=0x560991a847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 -- 172.21.15.16:0/2004493737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 msgr2=0x560991a7f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a7f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 -- 172.21.15.16:0/2004493737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560991965420 con 0x560991a84d30 2024-05-27T07:08:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.846+0000 7fbfcb400700 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a7e870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0095c0 tx=0x7fbfbc0098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7481dec2241e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.847+0000 7fbfc1600700 1 -- 172.21.15.16:0/2004493737 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc012a60 con 0x560991a84d30 2024-05-27T07:08:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.847+0000 7fbfc1600700 1 -- 172.21.15.16:0/2004493737 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfbc004cc0 con 0x560991a84d30 2024-05-27T07:08:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.847+0000 7fbfc1600700 1 -- 172.21.15.16:0/2004493737 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0054e0 con 0x560991a84d30 2024-05-27T07:08:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/2004493737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 msgr2=0x560991a7e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a7e870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0095c0 tx=0x7fbfbc0098d0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/2004493737 shutdown_connections 2024-05-27T07:08:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a7f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/2004493737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a7e870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.853+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/2004493737 >> 172.21.15.16:0/2004493737 conn(0x56099195ffc0 msgr2=0x560991a798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.854+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/2004493737 shutdown_connections 2024-05-27T07:08:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.854+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/2004493737 wait complete. 2024-05-27T07:08:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.855+0000 7fbfe1da3b80 1 -- start start 2024-05-27T07:08:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.856+0000 7fbfe1da3b80 1 Processor -- start 2024-05-27T07:08:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.858+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 0x560991a88560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991a88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609918a2fa0 con 0x5609918ffe40 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56099193d760 con 0x560991a7edb0 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560991a8b7a0 con 0x560991a84d30 2024-05-27T07:08:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.859+0000 7fbfe1da3b80 10 client.0 _ll_register_callbacks cb 0x560991946c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.860+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.860+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48242/0 (socket says 172.21.15.16:48242) 2024-05-27T07:08:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.860+0000 7fbfd1200700 1 -- 172.21.15.16:0/1395287483 learned_addr learned my addr 172.21.15.16:0/1395287483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.861+0000 7fbfcb400700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991a88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 0x560991a88560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfd1200700 1 -- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 msgr2=0x560991a88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfd1200700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991a88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfd1200700 1 -- 172.21.15.16:0/1395287483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 msgr2=0x560991a88560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfd1200700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 0x560991a88560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfd1200700 1 -- 172.21.15.16:0/1395287483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609918ec8d0 con 0x560991a84d30 2024-05-27T07:08:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.862+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 0x560991a88560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:08:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.863+0000 7fbfd1200700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40aac30 tx=0x7fbfc40e1890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ff9fa27297080a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.863+0000 7fbfba000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.864+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40ab630 con 0x560991a84d30 2024-05-27T07:08:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.864+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7fbfc40a7520 con 0x560991a84d30 2024-05-27T07:08:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.864+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc004cc0 con 0x560991a84d30 2024-05-27T07:08:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.865+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbfc40aaf30 con 0x560991a84d30 2024-05-27T07:08:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.865+0000 7fbfe1da3b80 10 client.26026 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.865+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56099196d060 con 0x560991a84d30 2024-05-27T07:08:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfbc004cc0 con 0x560991a84d30 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbfc40c1070 con 0x560991a84d30 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 10 client.26026 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 20 client.26026 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991a98710 0x560991a9ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 4 client.26026 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:08:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.866+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609919aed90 con 0x560991a98710 2024-05-27T07:08:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.867+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991a98710 0x560991a9ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.868+0000 7fbfba000700 10 client.26026 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:21.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.869+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991a98710 0x560991a9ab60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96840c2f1469f885 server_cookie=196e17262fabc05e in_seq=0 out_seq=0 2024-05-27T07:08:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.871+0000 7fbfba000700 1 -- 172.21.15.16:0/1395287483 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609919aed90 con 0x560991a98710 2024-05-27T07:08:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.871+0000 7fbfba000700 10 client.26026 handle_command_reply: tid=0 2024-05-27T07:08:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.871+0000 7fbfe1da3b80 1 client.26026 shutdown 2024-05-27T07:08:21.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 msgr2=0x560991a88fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40aac30 tx=0x7fbfc40e1890 comp rx=0 tx=0).stop 2024-05-27T07:08:21.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 shutdown_connections 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991a98710 0x560991a9ab60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a88fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfba000700 0 client.26026 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609918ffe40 0x560991a88560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/1395287483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991a88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.873+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 >> 172.21.15.16:0/1395287483 conn(0x56099195ffc0 msgr2=0x560991a79580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.875+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 shutdown_connections 2024-05-27T07:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.875+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/1395287483 wait complete. 2024-05-27T07:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.875+0000 7fbfe1da3b80 20 client.26026 trim_cache size 0 max 16384 2024-05-27T07:08:21.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.916+0000 7fbfe1da3b80 1 Processor -- start 2024-05-27T07:08:21.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 -- start start 2024-05-27T07:08:21.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:21 smithi016 ceph-mon[31415]: pgmap v229: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 376 op/s 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a87520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1ae470 con 0x560991a7edb0 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1affb0 con 0x5609918ffe40 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.917+0000 7fbfe1da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1aeb40 con 0x560991a84d30 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.918+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.918+0000 7fbfd1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:54358/0 (socket says 172.21.15.16:54358) 2024-05-27T07:08:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.918+0000 7fbfd1200700 1 -- 172.21.15.16:0/107058165 learned_addr learned my addr 172.21.15.16:0/107058165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 -- 172.21.15.16:0/107058165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 msgr2=0x560991a7e870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a7e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 -- 172.21.15.16:0/107058165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 msgr2=0x560991a87520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a87520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 -- 172.21.15.16:0/107058165 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560991a86ad0 con 0x560991a84d30 2024-05-27T07:08:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.919+0000 7fbfd1200700 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400b330 tx=0x7fbfc40aab60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c8888633d7b72f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.920+0000 7fbfc0c00700 1 -- 172.21.15.16:0/107058165 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40b3a80 con 0x560991a84d30 2024-05-27T07:08:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.920+0000 7fbfc0c00700 1 -- 172.21.15.16:0/107058165 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc40ab630 con 0x560991a84d30 2024-05-27T07:08:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.920+0000 7fbfc0c00700 1 -- 172.21.15.16:0/107058165 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40a68a0 con 0x560991a84d30 2024-05-27T07:08:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/107058165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 msgr2=0x560991a82b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400b330 tx=0x7fbfc40aab60 comp rx=0 tx=0).stop 2024-05-27T07:08:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/107058165 shutdown_connections 2024-05-27T07:08:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560991a84d30 0x560991a82b70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609918ffe40 0x560991a7e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/107058165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a7edb0 0x560991a87520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.921+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/107058165 >> 172.21.15.16:0/107058165 conn(0x56099195ffc0 msgr2=0x560991a88340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.922+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/107058165 shutdown_connections 2024-05-27T07:08:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.922+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/107058165 wait complete. 2024-05-27T07:08:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.923+0000 7fbfe1da3b80 1 -- start start 2024-05-27T07:08:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.923+0000 7fbfe1da3b80 1 Processor -- start 2024-05-27T07:08:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.926+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.926+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991aa0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.926+0000 7fbfe1da3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a98710 0x560991aa08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.928+0000 7fbfcbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.928+0000 7fbfcbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:54364/0 (socket says 172.21.15.16:54364) 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfcb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991aa0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfcb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991aa0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:54978/0 (socket says 172.21.15.16:54978) 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfcbe00700 1 -- 172.21.15.16:0/3182127981 learned_addr learned my addr 172.21.15.16:0/3182127981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1af8e0 con 0x560991a98710 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1af210 con 0x560991a7edb0 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1ac430 con 0x5609918ffe40 2024-05-27T07:08:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.929+0000 7fbfe1da3b80 10 client.0 _ll_register_callbacks cb 0x560991a842e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfd1200700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a98710 0x560991aa08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfcbe00700 1 -- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 msgr2=0x560991aa0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991aa0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfcbe00700 1 -- 172.21.15.16:0/3182127981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a98710 msgr2=0x560991aa08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a98710 0x560991aa08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.930+0000 7fbfcbe00700 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560991b02210 con 0x5609918ffe40 2024-05-27T07:08:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.932+0000 7fbfba000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.932+0000 7fbfcbe00700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560991a87de0 tx=0x7fbfb4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b07a8e7bbd245a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.933+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb402e060 con 0x5609918ffe40 2024-05-27T07:08:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.933+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7fbfb4004c40 con 0x5609918ffe40 2024-05-27T07:08:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.933+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc40ab630 con 0x5609918ffe40 2024-05-27T07:08:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.934+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbfb4035050 con 0x5609918ffe40 2024-05-27T07:08:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.934+0000 7fbfe1da3b80 10 client.26032 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.935+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560991b03440 con 0x5609918ffe40 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc40ab630 con 0x5609918ffe40 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbfb4003970 con 0x5609918ffe40 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfe1da3b80 10 client.26032 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfe1da3b80 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfe1da3b80 20 client.26032 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.936+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991b03a80 0x560991b05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:21.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.937+0000 7fbfe1da3b80 4 client.26032 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.937+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609919a8570 con 0x560991b03a80 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.937+0000 7fbfcb400700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991b03a80 0x560991b05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.938+0000 7fbfcb400700 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991b03a80 0x560991b05ea0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9aa1cc5981cf07 server_cookie=c5f7e32212e56962 in_seq=0 out_seq=0 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.938+0000 7fbfba000700 10 client.26032 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.939+0000 7fbfba000700 1 -- 172.21.15.16:0/3182127981 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5609919a8570 con 0x560991b03a80 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.939+0000 7fbfba000700 10 client.26032 handle_command_reply: tid=0 2024-05-27T07:08:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.939+0000 7fbfe1da3b80 1 client.26032 shutdown 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.939+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 msgr2=0x560991a9fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.939+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560991a87de0 tx=0x7fbfb4009310 comp rx=0 tx=0).stop 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 shutdown_connections 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560991b03a80 0x560991b05ea0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560991a98710 0x560991aa08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609918ffe40 0x560991a9fe50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 --2- 172.21.15.16:0/3182127981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560991a7edb0 0x560991aa0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 >> 172.21.15.16:0/3182127981 conn(0x56099195ffc0 msgr2=0x560991a828d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfba000700 0 client.26032 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:08:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 shutdown_connections 2024-05-27T07:08:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 1 -- 172.21.15.16:0/3182127981 wait complete. 2024-05-27T07:08:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.940+0000 7fbfe1da3b80 20 client.26032 trim_cache size 0 max 16384 2024-05-27T07:08:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:08:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:08:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:08:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:08:21.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 msgr2=0x7fbfb00b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 0x7fbfb00b50d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005e50 tx=0x7fbfb4005da0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 msgr2=0x7fbfcc1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:21.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009f60 tx=0x7fbfc400ce20 comp rx=0 tx=0).stop 2024-05-27T07:08:21.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 shutdown_connections 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbfb00b2bc0 0x7fbfb00b50d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfcc110b90 0x7fbfcc1a1750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc075d20 0x7fbfcc19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 --2- 172.21.15.16:0/3997569095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfcc076670 0x7fbfcc19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.942+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 >> 172.21.15.16:0/3997569095 conn(0x7fbfcc100030 msgr2=0x7fbfcc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.943+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 shutdown_connections 2024-05-27T07:08:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:21.943+0000 7fbfd3200700 1 -- 172.21.15.16:0/3997569095 wait complete. 2024-05-27T07:08:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:21 smithi164 ceph-mon[35327]: pgmap v229: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 376 op/s 2024-05-27T07:08:22.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:21 smithi069 ceph-mon[35276]: pgmap v229: 129 pgs: 129 active+clean; 16 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 376 op/s 2024-05-27T07:08:23.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:08:23.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:08:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:08:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:23 smithi164 ceph-mon[35327]: pgmap v230: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 MiB/s wr, 656 op/s 2024-05-27T07:08:24.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:23 smithi016 ceph-mon[31415]: pgmap v230: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 MiB/s wr, 656 op/s 2024-05-27T07:08:24.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:23 smithi069 ceph-mon[35276]: pgmap v230: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 MiB/s wr, 656 op/s 2024-05-27T07:08:26.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:25 smithi164 ceph-mon[35327]: pgmap v231: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 544 op/s 2024-05-27T07:08:26.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:25 smithi016 ceph-mon[31415]: pgmap v231: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 544 op/s 2024-05-27T07:08:26.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:25 smithi069 ceph-mon[35276]: pgmap v231: 129 pgs: 129 active+clean; 16 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 544 op/s 2024-05-27T07:08:27.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:27 smithi016 ceph-mon[31415]: pgmap v232: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:28.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:27 smithi164 ceph-mon[35327]: pgmap v232: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:28.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:27 smithi069 ceph-mon[35276]: pgmap v232: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:29 smithi164 ceph-mon[35327]: pgmap v233: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:30.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:29 smithi016 ceph-mon[31415]: pgmap v233: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:29 smithi069 ceph-mon[35276]: pgmap v233: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 560 op/s 2024-05-27T07:08:31.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:31 smithi016 ceph-mon[31415]: pgmap v234: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 31 MiB/s wr, 528 op/s 2024-05-27T07:08:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:31 smithi164 ceph-mon[35327]: pgmap v234: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 31 MiB/s wr, 528 op/s 2024-05-27T07:08:32.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:31 smithi069 ceph-mon[35276]: pgmap v234: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 31 MiB/s wr, 528 op/s 2024-05-27T07:08:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:33 smithi164 ceph-mon[35327]: pgmap v235: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 39 MiB/s wr, 670 op/s 2024-05-27T07:08:34.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:33 smithi016 ceph-mon[31415]: pgmap v235: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 39 MiB/s wr, 670 op/s 2024-05-27T07:08:34.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:33 smithi069 ceph-mon[35276]: pgmap v235: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 39 MiB/s wr, 670 op/s 2024-05-27T07:08:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.250+0000 7fd39e600700 1 -- 172.21.15.16:0/2993522894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398108f10 msgr2=0x7fd398109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.250+0000 7fd39e600700 1 --2- 172.21.15.16:0/2993522894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398108f10 0x7fd398109320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098f0 tx=0x7fd384009c00 comp rx=0 tx=0).stop 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.251+0000 7fd39e600700 1 -- 172.21.15.16:0/2993522894 shutdown_connections 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.251+0000 7fd39e600700 1 --2- 172.21.15.16:0/2993522894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd398109860 0x7fd3981111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.251+0000 7fd39e600700 1 --2- 172.21.15.16:0/2993522894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3981045e0 0x7fd3981049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.251+0000 7fd39e600700 1 --2- 172.21.15.16:0/2993522894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398108f10 0x7fd398109320 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098f0 tx=0x7fd384009c00 comp rx=0 tx=0).stop 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.251+0000 7fd39e600700 1 -- 172.21.15.16:0/2993522894 >> 172.21.15.16:0/2993522894 conn(0x7fd398100030 msgr2=0x7fd398102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.253+0000 7fd39e600700 1 -- 172.21.15.16:0/2993522894 shutdown_connections 2024-05-27T07:08:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.253+0000 7fd39e600700 1 -- 172.21.15.16:0/2993522894 wait complete. 2024-05-27T07:08:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.254+0000 7fd39e600700 1 Processor -- start 2024-05-27T07:08:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 -- start start 2024-05-27T07:08:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3981045e0 0x7fd398198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd39819d590 0x7fd39819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd398112210 con 0x7fd3981045e0 2024-05-27T07:08:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd398112090 con 0x7fd398109860 2024-05-27T07:08:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.255+0000 7fd39e600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd398112390 con 0x7fd39819d590 2024-05-27T07:08:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.256+0000 7fd396a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.256+0000 7fd396a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:49584/0 (socket says 172.21.15.16:49584) 2024-05-27T07:08:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.256+0000 7fd396a00700 1 -- 172.21.15.16:0/505502469 learned_addr learned my addr 172.21.15.16:0/505502469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.256+0000 7fd397400700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3981045e0 0x7fd398198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 -- 172.21.15.16:0/505502469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd39819d590 msgr2=0x7fd39819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd39819d590 0x7fd39819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 -- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3981045e0 msgr2=0x7fd398198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3981045e0 0x7fd398198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 -- 172.21.15.16:0/505502469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39819e0e0 con 0x7fd398109860 2024-05-27T07:08:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.257+0000 7fd396a00700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd384009810 tx=0x7fd38400fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.258+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840052e0 con 0x7fd398109860 2024-05-27T07:08:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.258+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd38400bc20 con 0x7fd398109860 2024-05-27T07:08:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.258+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384003e40 con 0x7fd398109860 2024-05-27T07:08:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.258+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39819e370 con 0x7fd398109860 2024-05-27T07:08:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.258+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3981a5a70 con 0x7fd398109860 2024-05-27T07:08:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.261+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3980512b0 con 0x7fd398109860 2024-05-27T07:08:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.267+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd38400b790 con 0x7fd398109860 2024-05-27T07:08:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.271+0000 7fd38fe00700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 0x7fd37c0b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.273+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7fd3840e3f00 con 0x7fd398109860 2024-05-27T07:08:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.273+0000 7fd397400700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 0x7fd37c0b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.274+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd384013030 con 0x7fd398109860 2024-05-27T07:08:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.277+0000 7fd397400700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 0x7fd37c0b9410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd390005e50 tx=0x7fd390005dc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.873+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39802e5e0 con 0x7fd398109860 2024-05-27T07:08:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.874+0000 7fd38fe00700 1 -- 172.21.15.16:0/505502469 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7fd3840ab530 con 0x7fd398109860 2024-05-27T07:08:34.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:08:34.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:08:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.878+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 msgr2=0x7fd37c0b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.878+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 0x7fd37c0b9410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd390005e50 tx=0x7fd390005dc0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.878+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 msgr2=0x7fd3981993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.878+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd384009810 tx=0x7fd38400fa90 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 shutdown_connections 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd37c0b6f00 0x7fd37c0b9410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd39819d590 0x7fd39819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3981045e0 0x7fd398198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 --2- 172.21.15.16:0/505502469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd398109860 0x7fd3981993c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.879+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 >> 172.21.15.16:0/505502469 conn(0x7fd398100030 msgr2=0x7fd398102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:34.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.880+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 shutdown_connections 2024-05-27T07:08:34.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:34.880+0000 7fd39e600700 1 -- 172.21.15.16:0/505502469 wait complete. 2024-05-27T07:08:34.878 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:08:35.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:34 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:08:35.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:34 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:08:35.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:34 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:08:36.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:35 smithi016 ceph-mon[31415]: pgmap v236: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 467 op/s 2024-05-27T07:08:36.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:35 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/505502469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:08:36.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:08:36.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:35 smithi164 ceph-mon[35327]: pgmap v236: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 467 op/s 2024-05-27T07:08:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:35 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/505502469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:08:36.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:35 smithi069 ceph-mon[35276]: pgmap v236: 129 pgs: 129 active+clean; 17 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 467 op/s 2024-05-27T07:08:36.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:35 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/505502469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:08:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:08:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:37 smithi016 ceph-mon[31415]: pgmap v237: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 40 MiB/s wr, 690 op/s 2024-05-27T07:08:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:37 smithi164 ceph-mon[35327]: pgmap v237: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 40 MiB/s wr, 690 op/s 2024-05-27T07:08:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:37 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:08:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:37 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:08:38.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:37 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:08:38.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:37 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:08:38.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:37 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:08:38.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:37 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:08:38.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:37 smithi069 ceph-mon[35276]: pgmap v237: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 40 MiB/s wr, 690 op/s 2024-05-27T07:08:38.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:37 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:08:38.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:37 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:08:38.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:37 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:08:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:39 smithi164 ceph-mon[35327]: pgmap v238: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:40.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:39 smithi016 ceph-mon[31415]: pgmap v238: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:40.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:39 smithi069 ceph-mon[35276]: pgmap v238: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:41 smithi164 ceph-mon[35327]: pgmap v239: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:41 smithi016 ceph-mon[31415]: pgmap v239: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:42.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:41 smithi069 ceph-mon[35276]: pgmap v239: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 472 op/s 2024-05-27T07:08:43.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:43 smithi016 ceph-mon[31415]: pgmap v240: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 38 MiB/s wr, 664 op/s 2024-05-27T07:08:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:43 smithi164 ceph-mon[35327]: pgmap v240: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 38 MiB/s wr, 664 op/s 2024-05-27T07:08:44.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:43 smithi069 ceph-mon[35276]: pgmap v240: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 38 MiB/s wr, 664 op/s 2024-05-27T07:08:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.630+0000 7f6569e00700 1 -- 172.21.15.16:0/1463314853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 msgr2=0x7f65641038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.630+0000 7f6569e00700 1 --2- 172.21.15.16:0/1463314853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f65641038c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f6554009cd0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.631+0000 7f6569e00700 1 -- 172.21.15.16:0/1463314853 shutdown_connections 2024-05-27T07:08:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.631+0000 7f6569e00700 1 --2- 172.21.15.16:0/1463314853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f6564110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.631+0000 7f6569e00700 1 --2- 172.21.15.16:0/1463314853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f65641038c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.631+0000 7f6569e00700 1 --2- 172.21.15.16:0/1463314853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f65641081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.631+0000 7f6569e00700 1 -- 172.21.15.16:0/1463314853 >> 172.21.15.16:0/1463314853 conn(0x7f65640fedb0 msgr2=0x7f65641011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.633+0000 7f6569e00700 1 -- 172.21.15.16:0/1463314853 shutdown_connections 2024-05-27T07:08:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.634+0000 7f6569e00700 1 -- 172.21.15.16:0/1463314853 wait complete. 2024-05-27T07:08:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.636+0000 7f6569e00700 1 Processor -- start 2024-05-27T07:08:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.637+0000 7f6569e00700 1 -- start start 2024-05-27T07:08:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.638+0000 7f6569e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f656410d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.638+0000 7f6569e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f656410d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.638+0000 7f6569e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.639+0000 7f6563400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.639+0000 7f6563400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42910/0 (socket says 172.21.15.16:42910) 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.639+0000 7f6563400700 1 -- 172.21.15.16:0/211962644 learned_addr learned my addr 172.21.15.16:0/211962644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.639+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65641121a0 con 0x7f6564108730 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.640+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6564112020 con 0x7f6564107de0 2024-05-27T07:08:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.640+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6564112320 con 0x7f65641034b0 2024-05-27T07:08:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.639+0000 7f6562000700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f656410d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.640+0000 7f6562a00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f656410d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.641+0000 7f6563400700 1 -- 172.21.15.16:0/211962644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 msgr2=0x7f656410d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.641+0000 7f6563400700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f656410d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.641+0000 7f6563400700 1 -- 172.21.15.16:0/211962644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 msgr2=0x7f656410d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.642+0000 7f6563400700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f656410d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.642+0000 7f6563400700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6564074220 con 0x7f6564108730 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.642+0000 7f6562000700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f656410d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:08:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.642+0000 7f6563400700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f655c007ee0 tx=0x7f655c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.643+0000 7f6568a00700 1 -- 172.21.15.16:0/211962644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c00ddd0 con 0x7f6564108730 2024-05-27T07:08:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.643+0000 7f6568a00700 1 -- 172.21.15.16:0/211962644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c004020 con 0x7f6564108730 2024-05-27T07:08:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.644+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6564074510 con 0x7f6564108730 2024-05-27T07:08:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.644+0000 7f6568a00700 1 -- 172.21.15.16:0/211962644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c015660 con 0x7f6564108730 2024-05-27T07:08:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.644+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6564074a50 con 0x7f6564108730 2024-05-27T07:08:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.654+0000 7f6568a00700 1 -- 172.21.15.16:0/211962644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f655c015800 con 0x7f6564108730 2024-05-27T07:08:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.657+0000 7f6568a00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 0x7f65480b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.658+0000 7f6568a00700 1 -- 172.21.15.16:0/211962644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f655c0e32f0 con 0x7f6564108730 2024-05-27T07:08:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.658+0000 7f6562a00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 0x7f65480b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.666+0000 7f6562a00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 0x7f65480b4fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f655401a040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.673+0000 7f6578a3eb80 1 Processor -- start 2024-05-27T07:08:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.673+0000 7f6578a3eb80 1 -- start start 2024-05-27T07:08:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.675+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.675+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b9d30 0x555b363b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b362e1e40 con 0x555b363b9d30 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555b362d19e0 con 0x555b363b3db0 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555b362d5a20 con 0x555b36234e40 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6562a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6562a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:35302/0 (socket says 172.21.15.16:35302) 2024-05-27T07:08:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.676+0000 7f6562a00700 1 -- 172.21.15.16:0/2980702600 learned_addr learned my addr 172.21.15.16:0/2980702600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 -- 172.21.15.16:0/2980702600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 msgr2=0x555b363b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 -- 172.21.15.16:0/2980702600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b9d30 msgr2=0x555b363b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b9d30 0x555b363b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 -- 172.21.15.16:0/2980702600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3629a420 con 0x555b36234e40 2024-05-27T07:08:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.677+0000 7f6562a00700 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f655401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35674e5a4cf5afbd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.678+0000 7f6558c00700 1 -- 172.21.15.16:0/2980702600 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554033070 con 0x555b36234e40 2024-05-27T07:08:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.678+0000 7f6558c00700 1 -- 172.21.15.16:0/2980702600 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65540056d0 con 0x555b36234e40 2024-05-27T07:08:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.678+0000 7f6558c00700 1 -- 172.21.15.16:0/2980702600 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655402e6d0 con 0x555b36234e40 2024-05-27T07:08:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.679+0000 7f6578a3eb80 1 -- 172.21.15.16:0/2980702600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 msgr2=0x555b363b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.679+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f655401e000 comp rx=0 tx=0).stop 2024-05-27T07:08:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.680+0000 7f6578a3eb80 1 -- 172.21.15.16:0/2980702600 shutdown_connections 2024-05-27T07:08:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.680+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.680+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363b97f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.680+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/2980702600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b9d30 0x555b363b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.680+0000 7f6578a3eb80 1 -- 172.21.15.16:0/2980702600 >> 172.21.15.16:0/2980702600 conn(0x555b36294fc0 msgr2=0x555b363ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.681+0000 7f6578a3eb80 1 -- 172.21.15.16:0/2980702600 shutdown_connections 2024-05-27T07:08:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.681+0000 7f6578a3eb80 1 -- 172.21.15.16:0/2980702600 wait complete. 2024-05-27T07:08:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.682+0000 7f6578a3eb80 1 -- start start 2024-05-27T07:08:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.683+0000 7f6578a3eb80 1 Processor -- start 2024-05-27T07:08:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 0x555b363bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b361d7fa0 con 0x555b363b3db0 2024-05-27T07:08:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555b36272760 con 0x555b36234e40 2024-05-27T07:08:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555b363c07a0 con 0x555b363b9d30 2024-05-27T07:08:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.684+0000 7f6578a3eb80 10 client.0 _ll_register_callbacks cb 0x555b3627bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.688+0000 7f6562a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 0x555b363bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.688+0000 7f6562a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 0x555b363bd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35646/0 (socket says 172.21.15.16:35646) 2024-05-27T07:08:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.688+0000 7f6562a00700 1 -- 172.21.15.16:0/276842222 learned_addr learned my addr 172.21.15.16:0/276842222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.688+0000 7f6563400700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.689+0000 7f6563400700 1 -- 172.21.15.16:0/276842222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 msgr2=0x555b363bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.689+0000 7f6563400700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 0x555b363bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.689+0000 7f6563400700 1 -- 172.21.15.16:0/276842222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 msgr2=0x555b363bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.689+0000 7f6563400700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.689+0000 7f6563400700 1 -- 172.21.15.16:0/276842222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b362218d0 con 0x555b363b9d30 2024-05-27T07:08:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.692+0000 7f6563400700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363be040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f655c0e0630 tx=0x7f655c0a9a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72cf910367367386 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.692+0000 7f654d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.693+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0ae5a0 con 0x555b363b9d30 2024-05-27T07:08:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.693+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f655c0ae160 con 0x555b363b9d30 2024-05-27T07:08:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.693+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65540056d0 con 0x555b363b9d30 2024-05-27T07:08:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.695+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f655c0ae360 con 0x555b363b9d30 2024-05-27T07:08:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.695+0000 7f6578a3eb80 10 client.26047 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.695+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b362a2060 con 0x555b363b9d30 2024-05-27T07:08:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.698+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65540056d0 con 0x555b363b9d30 2024-05-27T07:08:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.698+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f655c004bd0 con 0x555b363b9d30 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.699+0000 7f6578a3eb80 10 client.26047 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.699+0000 7f6578a3eb80 10 client.26047 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.699+0000 7f6578a3eb80 20 client.26047 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.700+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b363de740 0x555b363e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.700+0000 7f6578a3eb80 4 client.26047 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:08:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.700+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b362e3d90 con 0x555b363de740 2024-05-27T07:08:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.700+0000 7f6562a00700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b363de740 0x555b363e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.704+0000 7f6562a00700 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b363de740 0x555b363e0b60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=954597ba42c4b012 server_cookie=a01459b42fcbbb6c in_seq=0 out_seq=0 2024-05-27T07:08:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.704+0000 7f654d600700 10 client.26047 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:08:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.705+0000 7f654d600700 1 -- 172.21.15.16:0/276842222 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555b362e3d90 con 0x555b363de740 2024-05-27T07:08:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.705+0000 7f654d600700 10 client.26047 handle_command_reply: tid=0 2024-05-27T07:08:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.706+0000 7f6578a3eb80 1 client.26047 shutdown 2024-05-27T07:08:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.706+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 msgr2=0x555b363be040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.706+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363be040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f655c0e0630 tx=0x7f655c0a9a50 comp rx=0 tx=0).stop 2024-05-27T07:08:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 shutdown_connections 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b363de740 0x555b363e0b60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363be040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b36234e40 0x555b363bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/276842222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 >> 172.21.15.16:0/276842222 conn(0x555b36294fc0 msgr2=0x555b363ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f654d600700 0 client.26047 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 shutdown_connections 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 1 -- 172.21.15.16:0/276842222 wait complete. 2024-05-27T07:08:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.707+0000 7f6578a3eb80 20 client.26047 trim_cache size 0 max 16384 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.742+0000 7f6578a3eb80 1 Processor -- start 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.742+0000 7f6578a3eb80 1 -- start start 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.742+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b36234e40 0x555b363b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.742+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.742+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656411f060 con 0x555b36234e40 2024-05-27T07:08:44.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b88d0 con 0x555b363b3db0 2024-05-27T07:08:44.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b7500 con 0x555b363b9d30 2024-05-27T07:08:44.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6562a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b36234e40 0x555b363b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6562000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6562000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363bfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:35648/0 (socket says 172.21.15.16:35648) 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.743+0000 7f6562000700 1 -- 172.21.15.16:0/1231522403 learned_addr learned my addr 172.21.15.16:0/1231522403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 -- 172.21.15.16:0/1231522403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 msgr2=0x555b363bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 -- 172.21.15.16:0/1231522403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b36234e40 msgr2=0x555b363b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b36234e40 0x555b363b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.744+0000 7f6563400700 1 -- 172.21.15.16:0/1231522403 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b363e10c0 con 0x555b363b9d30 2024-05-27T07:08:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.745+0000 7f6563400700 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363bdab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f655c0aa180 tx=0x7f655c0aa370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aeff6601470f6ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.745+0000 7f654fe00700 1 -- 172.21.15.16:0/1231522403 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0ae5a0 con 0x555b363b9d30 2024-05-27T07:08:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.745+0000 7f654fe00700 1 -- 172.21.15.16:0/1231522403 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c0b3050 con 0x555b363b9d30 2024-05-27T07:08:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.747+0000 7f654fe00700 1 -- 172.21.15.16:0/1231522403 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c004bd0 con 0x555b363b9d30 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 -- 172.21.15.16:0/1231522403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 msgr2=0x555b363bdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363bdab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f655c0aa180 tx=0x7f655c0aa370 comp rx=0 tx=0).stop 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 -- 172.21.15.16:0/1231522403 shutdown_connections 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b363b9d30 0x555b363bdab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b36234e40 0x555b363b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/1231522403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b3db0 0x555b363bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 -- 172.21.15.16:0/1231522403 >> 172.21.15.16:0/1231522403 conn(0x555b36294fc0 msgr2=0x555b363bd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.748+0000 7f6578a3eb80 1 -- 172.21.15.16:0/1231522403 shutdown_connections 2024-05-27T07:08:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.749+0000 7f6578a3eb80 1 -- 172.21.15.16:0/1231522403 wait complete. 2024-05-27T07:08:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.750+0000 7f6578a3eb80 1 -- start start 2024-05-27T07:08:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.750+0000 7f6578a3eb80 1 Processor -- start 2024-05-27T07:08:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.752+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.752+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.753+0000 7f6578a3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 0x555b363c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.753+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b8200 con 0x555b363b3db0 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b7b30 con 0x555b363b9d30 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6578a3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f656411d020 con 0x555b36234e40 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6578a3eb80 10 client.0 _ll_register_callbacks cb 0x555b3627bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6563400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 0x555b363c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.753+0000 7f6562a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.753+0000 7f6562000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6562a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2024-05-27T07:08:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.754+0000 7f6562a00700 1 -- 172.21.15.16:0/3383542216 learned_addr learned my addr 172.21.15.16:0/3383542216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.759+0000 7f6563400700 1 -- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 msgr2=0x555b363e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.759+0000 7f6563400700 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.759+0000 7f6563400700 1 -- 172.21.15.16:0/3383542216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 msgr2=0x555b363c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.759+0000 7f6563400700 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.759+0000 7f6563400700 1 -- 172.21.15.16:0/3383542216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b363e52f0 con 0x555b363b9d30 2024-05-27T07:08:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.760+0000 7f6563400700 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 0x555b363c5ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x555b363c0010 tx=0x7f655c0b2df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7304f12aca21d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.760+0000 7f654d600700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:08:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.761+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0ae5a0 con 0x555b363b9d30 2024-05-27T07:08:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.761+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f655c0ae160 con 0x555b363b9d30 2024-05-27T07:08:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.763+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c0b3050 con 0x555b363b9d30 2024-05-27T07:08:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.763+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f655c0a9450 con 0x555b363b9d30 2024-05-27T07:08:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.763+0000 7f6578a3eb80 10 client.36110 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.763+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b364483e0 con 0x555b363b9d30 2024-05-27T07:08:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.764+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c0b3050 con 0x555b363b9d30 2024-05-27T07:08:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f655c0a5c60 con 0x555b363b9d30 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 10 client.36110 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 10 client.36110 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 20 client.36110 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b36448980 0x555b3644ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 4 client.36110 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:08:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.766+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b362dd570 con 0x555b36448980 2024-05-27T07:08:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.770+0000 7f6562a00700 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b36448980 0x555b3644ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:44.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.774+0000 7f6562a00700 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b36448980 0x555b3644ada0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98b990041a58155b server_cookie=b904a5805c0b639e in_seq=0 out_seq=0 2024-05-27T07:08:44.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.774+0000 7f654d600700 10 client.36110 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:08:44.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f654d600700 1 -- 172.21.15.16:0/3383542216 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6554041020 con 0x555b36448980 2024-05-27T07:08:44.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f654d600700 10 client.36110 handle_command_reply: tid=0 2024-05-27T07:08:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 client.36110 shutdown 2024-05-27T07:08:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 msgr2=0x555b363c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 0x555b363c5ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x555b363c0010 tx=0x7f655c0b2df0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.768 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:08:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 shutdown_connections 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x555b36448980 0x555b3644ada0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555b363b9d30 0x555b363c5ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b36234e40 0x555b363e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 --2- 172.21.15.16:0/3383542216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b363b3db0 0x555b363c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.775+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 >> 172.21.15.16:0/3383542216 conn(0x555b36294fc0 msgr2=0x555b363bddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.776+0000 7f654d600700 0 client.36110 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.776+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 shutdown_connections 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.776+0000 7f6578a3eb80 1 -- 172.21.15.16:0/3383542216 wait complete. 2024-05-27T07:08:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.776+0000 7f6578a3eb80 20 client.36110 trim_cache size 0 max 16384 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.777+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 msgr2=0x7f65480b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 0x7f65480b4fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f655401a040 comp rx=0 tx=0).stop 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 msgr2=0x7f656410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f655c007ee0 tx=0x7f655c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 shutdown_connections 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f65480b2aa0 0x7f65480b4fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108730 0x7f656410dd70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65641034b0 0x7f656410d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 --2- 172.21.15.16:0/211962644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564107de0 0x7f656410d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.778+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 >> 172.21.15.16:0/211962644 conn(0x7f65640fedb0 msgr2=0x7f6564109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.779+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 shutdown_connections 2024-05-27T07:08:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:44.780+0000 7f6569e00700 1 -- 172.21.15.16:0/211962644 wait complete. 2024-05-27T07:08:45.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:08:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:45 smithi016 ceph-mon[31415]: pgmap v241: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 460 op/s 2024-05-27T07:08:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:45 smithi164 ceph-mon[35327]: pgmap v241: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 460 op/s 2024-05-27T07:08:46.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:45 smithi069 ceph-mon[35276]: pgmap v241: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 460 op/s 2024-05-27T07:08:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:08:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:47 smithi164 ceph-mon[35327]: pgmap v242: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 37 MiB/s wr, 650 op/s 2024-05-27T07:08:48.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:47 smithi016 ceph-mon[31415]: pgmap v242: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 37 MiB/s wr, 650 op/s 2024-05-27T07:08:48.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:47 smithi069 ceph-mon[35276]: pgmap v242: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 37 MiB/s wr, 650 op/s 2024-05-27T07:08:50.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:49 smithi164 ceph-mon[35327]: pgmap v243: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-27T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:49 smithi016 ceph-mon[31415]: pgmap v243: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-27T07:08:50.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:49 smithi069 ceph-mon[35276]: pgmap v243: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-27T07:08:52.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:51 smithi016 ceph-mon[31415]: pgmap v244: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 449 op/s 2024-05-27T07:08:52.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:51 smithi164 ceph-mon[35327]: pgmap v244: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 449 op/s 2024-05-27T07:08:52.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:51 smithi069 ceph-mon[35276]: pgmap v244: 129 pgs: 129 active+clean; 17 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 449 op/s 2024-05-27T07:08:54.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:53 smithi164 ceph-mon[35327]: pgmap v245: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 39 MiB/s wr, 685 op/s 2024-05-27T07:08:54.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:53 smithi016 ceph-mon[31415]: pgmap v245: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 39 MiB/s wr, 685 op/s 2024-05-27T07:08:54.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:53 smithi069 ceph-mon[35276]: pgmap v245: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 39 MiB/s wr, 685 op/s 2024-05-27T07:08:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 -- 172.21.15.16:0/332543198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c10a920 msgr2=0x7f691c10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 --2- 172.21.15.16:0/332543198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c10a920 0x7f691c10ad30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f690c009990 tx=0x7f690c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 -- 172.21.15.16:0/332543198 shutdown_connections 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 --2- 172.21.15.16:0/332543198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c11d920 0x7f691c11fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 --2- 172.21.15.16:0/332543198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c10a920 0x7f691c10ad30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 --2- 172.21.15.16:0/332543198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10b270 0x7f691c11d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.946+0000 7f6922a00700 1 -- 172.21.15.16:0/332543198 >> 172.21.15.16:0/332543198 conn(0x7f691c06f4c0 msgr2=0x7f691c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.949+0000 7f6922a00700 1 -- 172.21.15.16:0/332543198 shutdown_connections 2024-05-27T07:08:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.949+0000 7f6922a00700 1 -- 172.21.15.16:0/332543198 wait complete. 2024-05-27T07:08:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.950+0000 7f6922a00700 1 Processor -- start 2024-05-27T07:08:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.951+0000 7f6922a00700 1 -- start start 2024-05-27T07:08:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.951+0000 7f6922a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 0x7f691c1af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.951+0000 7f6922a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.951+0000 7f6922a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 0x7f691c1a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.952+0000 7f691b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.952+0000 7f691b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44360/0 (socket says 172.21.15.16:44360) 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.952+0000 7f691be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 0x7f691c1af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.952+0000 7f6920a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 0x7f691c1a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.952+0000 7f6920a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 0x7f691c1a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51758/0 (socket says 172.21.15.16:51758) 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f691be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 0x7f691c1af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48110/0 (socket says 172.21.15.16:48110) 2024-05-27T07:08:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f691b400700 1 -- 172.21.15.16:0/3200889061 learned_addr learned my addr 172.21.15.16:0/3200889061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1231d0 con 0x7f691c10a920 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f691c123050 con 0x7f691c11d920 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f691c123350 con 0x7f691c10b270 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f691b400700 1 -- 172.21.15.16:0/3200889061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 msgr2=0x7f691c1a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.953+0000 7f691b400700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 0x7f691c1a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.954+0000 7f691b400700 1 -- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 msgr2=0x7f691c1af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.954+0000 7f691b400700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 0x7f691c1af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.954+0000 7f691b400700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c1aa0c0 con 0x7f691c10b270 2024-05-27T07:08:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.955+0000 7f691b400700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9a0 tx=0x7f690800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.958+0000 7f6918c00700 1 -- 172.21.15.16:0/3200889061 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690800cbd0 con 0x7f691c10b270 2024-05-27T07:08:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.958+0000 7f6918c00700 1 -- 172.21.15.16:0/3200889061 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f690800cd70 con 0x7f691c10b270 2024-05-27T07:08:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.960+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1aa3b0 con 0x7f691c10b270 2024-05-27T07:08:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.960+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1b6680 con 0x7f691c10b270 2024-05-27T07:08:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.961+0000 7f6918c00700 1 -- 172.21.15.16:0/3200889061 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908018680 con 0x7f691c10b270 2024-05-27T07:08:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.970+0000 7f6918c00700 1 -- 172.21.15.16:0/3200889061 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6908010430 con 0x7f691c10b270 2024-05-27T07:08:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.977+0000 7f6918c00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 0x7f69000b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.977+0000 7f6918c00700 1 -- 172.21.15.16:0/3200889061 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f69080e2ac0 con 0x7f691c10b270 2024-05-27T07:08:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.978+0000 7f691be00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 0x7f69000b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.981+0000 7f691be00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 0x7f69000b4fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f690c00b2e0 tx=0x7f690c005fd0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.985+0000 7f69315c3b80 1 Processor -- start 2024-05-27T07:08:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 -- start start 2024-05-27T07:08:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 0x55acd73ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd73e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55acd7317e40 con 0x55acd726ae40 2024-05-27T07:08:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acd73079e0 con 0x55acd73efd30 2024-05-27T07:08:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.986+0000 7f69315c3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acd730ba20 con 0x55acd73e9db0 2024-05-27T07:08:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.988+0000 7f691b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd73e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.988+0000 7f6920a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.989+0000 7f6920a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:44366/0 (socket says 172.21.15.16:44366) 2024-05-27T07:08:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.989+0000 7f6920a00700 1 -- 172.21.15.16:0/1901521478 learned_addr learned my addr 172.21.15.16:0/1901521478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.989+0000 7f691be00700 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 0x55acd73ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.990+0000 7f6920a00700 1 -- 172.21.15.16:0/1901521478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 msgr2=0x55acd73e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.990+0000 7f6920a00700 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd73e9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.990+0000 7f6920a00700 1 -- 172.21.15.16:0/1901521478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 msgr2=0x55acd73ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.991+0000 7f6920a00700 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 0x55acd73ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.991+0000 7f6920a00700 1 -- 172.21.15.16:0/1901521478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd72d0420 con 0x55acd73e9db0 2024-05-27T07:08:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.992+0000 7f691be00700 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 0x55acd73ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:08:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.993+0000 7f6920a00700 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69140115e0 tx=0x7f6914009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08d7bb8ef39375c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.993+0000 7f6911600700 1 -- 172.21.15.16:0/1901521478 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914019030 con 0x55acd73e9db0 2024-05-27T07:08:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.993+0000 7f6911600700 1 -- 172.21.15.16:0/1901521478 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6914009320 con 0x55acd73e9db0 2024-05-27T07:08:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.993+0000 7f6911600700 1 -- 172.21.15.16:0/1901521478 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69140047e0 con 0x55acd73e9db0 2024-05-27T07:08:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.995+0000 7f69315c3b80 1 -- 172.21.15.16:0/1901521478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 msgr2=0x55acd73ea1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.995+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69140115e0 tx=0x7f6914009920 comp rx=0 tx=0).stop 2024-05-27T07:08:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.996+0000 7f69315c3b80 1 -- 172.21.15.16:0/1901521478 shutdown_connections 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73e9db0 0x55acd73ea1c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd726ae40 0x55acd73ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1901521478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd73e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 -- 172.21.15.16:0/1901521478 >> 172.21.15.16:0/1901521478 conn(0x55acd72cafc0 msgr2=0x55acd73e48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 -- 172.21.15.16:0/1901521478 shutdown_connections 2024-05-27T07:08:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.997+0000 7f69315c3b80 1 -- 172.21.15.16:0/1901521478 wait complete. 2024-05-27T07:08:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.998+0000 7f69315c3b80 1 -- start start 2024-05-27T07:08:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.998+0000 7f69315c3b80 1 Processor -- start 2024-05-27T07:08:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.999+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.999+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:55.999+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f69315c3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55acd720dfa0 con 0x55acd73e9db0 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f69315c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acd72a8760 con 0x55acd726ae40 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f69315c3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acd73f6800 con 0x55acd73efd30 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f691be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f691be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51788/0 (socket says 172.21.15.16:51788) 2024-05-27T07:08:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f691be00700 1 -- 172.21.15.16:0/1068346912 learned_addr learned my addr 172.21.15.16:0/1068346912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.001+0000 7f6920a00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.002+0000 7f691b400700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.002+0000 7f6920a00700 1 -- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 msgr2=0x55acd73f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.002+0000 7f6920a00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.002+0000 7f6920a00700 1 -- 172.21.15.16:0/1068346912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 msgr2=0x55acd73f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.003+0000 7f6920a00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.003+0000 7f6920a00700 1 -- 172.21.15.16:0/1068346912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd72578d0 con 0x55acd73efd30 2024-05-27T07:08:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.003+0000 7f691b400700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:08:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.005+0000 7f69315c3b80 10 client.0 _ll_register_callbacks cb 0x55acd72b1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.005+0000 7f6920a00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f4020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6914000f80 tx=0x7f6914003ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97078892833ffa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.005+0000 7f6906000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:08:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.005+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6914009320 con 0x55acd73efd30 2024-05-27T07:08:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.006+0000 7f6906000700 1 -- 172.21.15.16:0/1068346912 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914023070 con 0x55acd73efd30 2024-05-27T07:08:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.006+0000 7f6906000700 1 -- 172.21.15.16:0/1068346912 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f6914004790 con 0x55acd73efd30 2024-05-27T07:08:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.006+0000 7f6906000700 1 -- 172.21.15.16:0/1068346912 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6914007500 con 0x55acd73efd30 2024-05-27T07:08:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.006+0000 7f69315c3b80 10 client.26065 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.006+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55acd72d8060 con 0x55acd73efd30 2024-05-27T07:08:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.007+0000 7f6906000700 1 -- 172.21.15.16:0/1068346912 <== mon.1 v2:172.21.15.164:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f69140076a0 con 0x55acd73efd30 2024-05-27T07:08:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.009+0000 7f69315c3b80 10 client.26065 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.010+0000 7f69315c3b80 10 client.26065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:08:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.010+0000 7f69315c3b80 20 client.26065 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.010+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd74033b0 0x55acd74057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.010+0000 7f69315c3b80 4 client.26065 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:08:56.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.010+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd7319d90 con 0x55acd74033b0 2024-05-27T07:08:56.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.011+0000 7f691be00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd74033b0 0x55acd74057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.013+0000 7f6906000700 10 client.26065 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:08:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.013+0000 7f691be00700 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd74033b0 0x55acd74057d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11de4820e92a1085 server_cookie=fecc20569b91b5a5 in_seq=0 out_seq=0 2024-05-27T07:08:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.015+0000 7f6906000700 1 -- 172.21.15.16:0/1068346912 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55acd7319d90 con 0x55acd74033b0 2024-05-27T07:08:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.015+0000 7f6906000700 10 client.26065 handle_command_reply: tid=0 2024-05-27T07:08:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.015+0000 7f69315c3b80 1 client.26065 shutdown 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.015+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 msgr2=0x55acd73f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.015+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f4020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6914000f80 tx=0x7f6914003ca0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 shutdown_connections 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f4020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd74033b0 0x55acd74057d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1068346912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 >> 172.21.15.16:0/1068346912 conn(0x55acd72cafc0 msgr2=0x55acd73e4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.016+0000 7f6906000700 0 client.26065 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.017+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 shutdown_connections 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.017+0000 7f69315c3b80 1 -- 172.21.15.16:0/1068346912 wait complete. 2024-05-27T07:08:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.017+0000 7f69315c3b80 20 client.26065 trim_cache size 0 max 16384 2024-05-27T07:08:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 Processor -- start 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 -- start start 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1bf290 con 0x55acd73e9db0 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1c0dd0 con 0x55acd726ae40 2024-05-27T07:08:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.059+0000 7f69315c3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1bf960 con 0x55acd73efd30 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.060+0000 7f691be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.060+0000 7f6920a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.060+0000 7f691be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51804/0 (socket says 172.21.15.16:51804) 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.060+0000 7f691be00700 1 -- 172.21.15.16:0/4036844493 learned_addr learned my addr 172.21.15.16:0/4036844493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 -- 172.21.15.16:0/4036844493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 msgr2=0x55acd73f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 -- 172.21.15.16:0/4036844493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 msgr2=0x55acd73eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 -- 172.21.15.16:0/4036844493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd73e6280 con 0x55acd726ae40 2024-05-27T07:08:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.061+0000 7f691be00700 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f690c0098e0 tx=0x7f690c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2d904289ff6392 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.062+0000 7f6910c00700 1 -- 172.21.15.16:0/4036844493 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00be30 con 0x55acd726ae40 2024-05-27T07:08:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.064+0000 7f6910c00700 1 -- 172.21.15.16:0/4036844493 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f690c003b10 con 0x55acd726ae40 2024-05-27T07:08:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.064+0000 7f6910c00700 1 -- 172.21.15.16:0/4036844493 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c032a60 con 0x55acd726ae40 2024-05-27T07:08:56.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.066+0000 7f69315c3b80 1 -- 172.21.15.16:0/4036844493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 msgr2=0x55acd73e9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.066+0000 7f69315c3b80 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f690c0098e0 tx=0x7f690c020000 comp rx=0 tx=0).stop 2024-05-27T07:08:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.067+0000 7f69315c3b80 1 -- 172.21.15.16:0/4036844493 shutdown_connections 2024-05-27T07:08:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.067+0000 7f69315c3b80 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd73efd30 0x55acd73f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.067+0000 7f69315c3b80 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd726ae40 0x55acd73e9870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.067+0000 7f69315c3b80 1 --2- 172.21.15.16:0/4036844493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd73eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.067+0000 7f69315c3b80 1 -- 172.21.15.16:0/4036844493 >> 172.21.15.16:0/4036844493 conn(0x55acd72cafc0 msgr2=0x55acd73f1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.069+0000 7f69315c3b80 1 -- 172.21.15.16:0/4036844493 shutdown_connections 2024-05-27T07:08:56.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:55 smithi016 ceph-mon[31415]: pgmap v246: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 461 op/s 2024-05-27T07:08:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.069+0000 7f69315c3b80 1 -- 172.21.15.16:0/4036844493 wait complete. 2024-05-27T07:08:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.070+0000 7f69315c3b80 1 -- start start 2024-05-27T07:08:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.071+0000 7f69315c3b80 1 Processor -- start 2024-05-27T07:08:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.072+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 0x55acd74053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.072+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd7405900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.072+0000 7f69315c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f69315c3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1c0700 con 0x55acd73e9db0 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f69315c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1c0030 con 0x55acd73efd30 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f69315c3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1bd250 con 0x55acd726ae40 2024-05-27T07:08:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f69315c3b80 10 client.0 _ll_register_callbacks cb 0x55acd72b1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:08:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f6920a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f6920a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51814/0 (socket says 172.21.15.16:51814) 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.075+0000 7f6920a00700 1 -- 172.21.15.16:0/1015441966 learned_addr learned my addr 172.21.15.16:0/1015441966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f691b400700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd7405900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f6920a00700 1 -- 172.21.15.16:0/1015441966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 msgr2=0x55acd74053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f691be00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 0x55acd74053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f6920a00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 0x55acd74053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f6920a00700 1 -- 172.21.15.16:0/1015441966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 msgr2=0x55acd7405900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f6920a00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd7405900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.076+0000 7f6920a00700 1 -- 172.21.15.16:0/1015441966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd747cc20 con 0x55acd73efd30 2024-05-27T07:08:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.077+0000 7f691be00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 0x55acd74053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:08:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.077+0000 7f6920a00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ec0 tx=0x7f691400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d9e882def9abe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:08:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.077+0000 7f6906000700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:08:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.078+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914019040 con 0x55acd73efd30 2024-05-27T07:08:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.078+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9509+0+0 (secure 0 0 0) 0x7f6914004790 con 0x55acd73efd30 2024-05-27T07:08:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.079+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f690c003b10 con 0x55acd73efd30 2024-05-27T07:08:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.080+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6914004990 con 0x55acd73efd30 2024-05-27T07:08:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.083+0000 7f69315c3b80 10 client.36128 fetch_fsmap learned FSMap version 31 2024-05-27T07:08:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.083+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55acd747dd40 con 0x55acd73efd30 2024-05-27T07:08:56.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.084+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f690c003b10 con 0x55acd73efd30 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.084+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f691403c890 con 0x55acd73efd30 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 10 client.36128 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 10 client.36128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 20 client.36128 populate_metadata read hostname 'smithi016' 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd747e340 0x55acd7480760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 4 client.36128 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:08:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.085+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55acd7313570 con 0x55acd747e340 2024-05-27T07:08:56.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.088+0000 7f691be00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd747e340 0x55acd7480760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:08:56.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:55 smithi164 ceph-mon[35327]: pgmap v246: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 461 op/s 2024-05-27T07:08:56.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.089+0000 7f691be00700 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd747e340 0x55acd7480760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa09f6fae047a82b server_cookie=736bcc9486dadf10 in_seq=0 out_seq=0 2024-05-27T07:08:56.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.089+0000 7f6906000700 10 client.36128 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:08:56.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f6906000700 1 -- 172.21.15.16:0/1015441966 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55acd7313570 con 0x55acd747e340 2024-05-27T07:08:56.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f6906000700 10 client.36128 handle_command_reply: tid=0 2024-05-27T07:08:56.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 client.36128 shutdown 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 msgr2=0x55acd7405e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ec0 tx=0x7f691400b040 comp rx=0 tx=0).stop 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 shutdown_connections 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55acd747e340 0x55acd7480760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd73efd30 0x55acd7405e40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acd726ae40 0x55acd74053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.090+0000 7f69315c3b80 1 --2- 172.21.15.16:0/1015441966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acd73e9db0 0x55acd7405900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.091+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 >> 172.21.15.16:0/1015441966 conn(0x55acd72cafc0 msgr2=0x55acd73ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:56.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.091+0000 7f6906000700 0 client.36128 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:08:56.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.093+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 shutdown_connections 2024-05-27T07:08:56.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.093+0000 7f69315c3b80 1 -- 172.21.15.16:0/1015441966 wait complete. 2024-05-27T07:08:56.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.095+0000 7f69315c3b80 20 client.36128 trim_cache size 0 max 16384 2024-05-27T07:08:56.087 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 msgr2=0x7f69000b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 0x7f69000b4fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f690c00b2e0 tx=0x7f690c005fd0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 msgr2=0x7f691c1afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9a0 tx=0x7f690800ecb0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 shutdown_connections 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f69000b2ad0 0x7f69000b4fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c11d920 0x7f691c1a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c10a920 0x7f691c1af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 --2- 172.21.15.16:0/3200889061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f691c10b270 0x7f691c1afd30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.097+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 >> 172.21.15.16:0/3200889061 conn(0x7f691c06f4c0 msgr2=0x7f691c11b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.098+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 shutdown_connections 2024-05-27T07:08:56.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:08:56.098+0000 7f6922a00700 1 -- 172.21.15.16:0/3200889061 wait complete. 2024-05-27T07:08:56.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:55 smithi069 ceph-mon[35276]: pgmap v246: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 461 op/s 2024-05-27T07:08:57.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:08:58.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:57 smithi164 ceph-mon[35327]: pgmap v247: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 37 MiB/s wr, 660 op/s 2024-05-27T07:08:58.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:57 smithi016 ceph-mon[31415]: pgmap v247: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 37 MiB/s wr, 660 op/s 2024-05-27T07:08:58.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:57 smithi069 ceph-mon[35276]: pgmap v247: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 37 MiB/s wr, 660 op/s 2024-05-27T07:08:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:00.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:08:59 smithi164 ceph-mon[35327]: pgmap v248: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 29 MiB/s wr, 510 op/s 2024-05-27T07:09:00.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:08:59 smithi016 ceph-mon[31415]: pgmap v248: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 29 MiB/s wr, 510 op/s 2024-05-27T07:09:00.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:08:59 smithi069 ceph-mon[35276]: pgmap v248: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 29 MiB/s wr, 510 op/s 2024-05-27T07:09:02.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:01 smithi016 ceph-mon[31415]: pgmap v249: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 474 op/s 2024-05-27T07:09:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:01 smithi069 ceph-mon[35276]: pgmap v249: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 474 op/s 2024-05-27T07:09:02.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:01 smithi164 ceph-mon[35327]: pgmap v249: 129 pgs: 129 active+clean; 17 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 474 op/s 2024-05-27T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:03 smithi016 ceph-mon[31415]: pgmap v250: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 39 MiB/s wr, 706 op/s 2024-05-27T07:09:04.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:03 smithi069 ceph-mon[35276]: pgmap v250: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 39 MiB/s wr, 706 op/s 2024-05-27T07:09:04.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:03 smithi164 ceph-mon[35327]: pgmap v250: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 39 MiB/s wr, 706 op/s 2024-05-27T07:09:06.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:05 smithi016 ceph-mon[31415]: pgmap v251: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 471 op/s 2024-05-27T07:09:06.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:05 smithi069 ceph-mon[35276]: pgmap v251: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 471 op/s 2024-05-27T07:09:06.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:05 smithi164 ceph-mon[35327]: pgmap v251: 129 pgs: 129 active+clean; 18 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 471 op/s 2024-05-27T07:09:08.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:07 smithi069 ceph-mon[35276]: pgmap v252: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 35 MiB/s wr, 651 op/s 2024-05-27T07:09:08.181 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:07 smithi016 ceph-mon[31415]: pgmap v252: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 35 MiB/s wr, 651 op/s 2024-05-27T07:09:08.335 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:07 smithi164 ceph-mon[35327]: pgmap v252: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 35 MiB/s wr, 651 op/s 2024-05-27T07:09:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.293+0000 7f00e6000700 1 -- 172.21.15.16:0/1215904043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0075b50 msgr2=0x7f00e0075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.293+0000 7f00e6000700 1 --2- 172.21.15.16:0/1215904043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0075b50 0x7f00e0075f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009f90 tx=0x7f00d800b330 comp rx=0 tx=0).stop 2024-05-27T07:09:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 -- 172.21.15.16:0/1215904043 shutdown_connections 2024-05-27T07:09:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 --2- 172.21.15.16:0/1215904043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 0x7f00e0117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 --2- 172.21.15.16:0/1215904043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0075b50 0x7f00e0075f60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 --2- 172.21.15.16:0/1215904043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e00764a0 0x7f00e0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 -- 172.21.15.16:0/1215904043 >> 172.21.15.16:0/1215904043 conn(0x7f00e0100030 msgr2=0x7f00e0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.294+0000 7f00e6000700 1 -- 172.21.15.16:0/1215904043 shutdown_connections 2024-05-27T07:09:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.295+0000 7f00e6000700 1 -- 172.21.15.16:0/1215904043 wait complete. 2024-05-27T07:09:09.288 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:08 smithi016 ceph-mon[31415]: pgmap v253: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 26 MiB/s wr, 497 op/s 2024-05-27T07:09:09.288 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:08 smithi016 ceph-mon[31415]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:09:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.297+0000 7f00e6000700 1 Processor -- start 2024-05-27T07:09:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.299+0000 7f00e6000700 1 -- start start 2024-05-27T07:09:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.300+0000 7f00e6000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e0075b50 0x7f00e01a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.300+0000 7f00e6000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.300+0000 7f00e6000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 0x7f00e01a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00e6000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00e011a960 con 0x7f00e0075b50 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00e6000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00e011a7e0 con 0x7f00e0115360 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00e6000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e011aae0 con 0x7f00e00764a0 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00dea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00df400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e0075b50 0x7f00e01a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00dea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47240/0 (socket says 172.21.15.16:47240) 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00dea00700 1 -- 172.21.15.16:0/792014819 learned_addr learned my addr 172.21.15.16:0/792014819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.301+0000 7f00dfe00700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 0x7f00e01a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.302+0000 7f00dea00700 1 -- 172.21.15.16:0/792014819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 msgr2=0x7f00e01a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.302+0000 7f00dea00700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 0x7f00e01a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.302+0000 7f00dea00700 1 -- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e0075b50 msgr2=0x7f00e01a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.302+0000 7f00dea00700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e0075b50 0x7f00e01a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.302+0000 7f00dea00700 1 -- 172.21.15.16:0/792014819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e01a6260 con 0x7f00e00764a0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.303+0000 7f00dea00700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f00d000e9a0 tx=0x7f00d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.303+0000 7f00d7e00700 1 -- 172.21.15.16:0/792014819 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000ca10 con 0x7f00e00764a0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.303+0000 7f00d7e00700 1 -- 172.21.15.16:0/792014819 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00d000cbb0 con 0x7f00e00764a0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.303+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01a6550 con 0x7f00e00764a0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.303+0000 7f00d7e00700 1 -- 172.21.15.16:0/792014819 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0018680 con 0x7f00e00764a0 2024-05-27T07:09:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.304+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01ade70 con 0x7f00e00764a0 2024-05-27T07:09:09.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.311+0000 7f00d7e00700 1 -- 172.21.15.16:0/792014819 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f00d0018820 con 0x7f00e00764a0 2024-05-27T07:09:09.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.314+0000 7f00d7e00700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 0x7f00c40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.314+0000 7f00d7e00700 1 -- 172.21.15.16:0/792014819 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9613+0+0 (secure 0 0 0) 0x7f00d00e2e60 con 0x7f00e00764a0 2024-05-27T07:09:09.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.317+0000 7f00df400700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 0x7f00c40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.318+0000 7f00df400700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 0x7f00c40b5000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009da0 tx=0x7f00d80096c0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.335+0000 7f00f4ac1b80 1 Processor -- start 2024-05-27T07:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.335+0000 7f00f4ac1b80 1 -- start start 2024-05-27T07:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.336+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10cd8e40 0x560d10e5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.336+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e57870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.336+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.336+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d10d85e40 con 0x560d10cd8e40 2024-05-27T07:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d10d759e0 con 0x560d10e57db0 2024-05-27T07:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d10d79a20 con 0x560d10e5dd30 2024-05-27T07:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:59914/0 (socket says 172.21.15.16:59914) 2024-05-27T07:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00dfe00700 1 -- 172.21.15.16:0/3683892560 learned_addr learned my addr 172.21.15.16:0/3683892560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.337+0000 7f00dea00700 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e57870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.338+0000 7f00dfe00700 1 -- 172.21.15.16:0/3683892560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 msgr2=0x560d10e57870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.338+0000 7f00dfe00700 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e57870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.338+0000 7f00dfe00700 1 -- 172.21.15.16:0/3683892560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10cd8e40 msgr2=0x560d10e5d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.338+0000 7f00dfe00700 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10cd8e40 0x560d10e5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.338+0000 7f00dfe00700 1 -- 172.21.15.16:0/3683892560 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d10d3e420 con 0x560d10e57db0 2024-05-27T07:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.339+0000 7f00dfe00700 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0046a0 tx=0x7f00cc00b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e7343b869149ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.340+0000 7f00d4c00700 1 -- 172.21.15.16:0/3683892560 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc003940 con 0x560d10e57db0 2024-05-27T07:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.340+0000 7f00d4c00700 1 -- 172.21.15.16:0/3683892560 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00cc015070 con 0x560d10e57db0 2024-05-27T07:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.340+0000 7f00d4c00700 1 -- 172.21.15.16:0/3683892560 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc01ae40 con 0x560d10e57db0 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.341+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3683892560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 msgr2=0x560d10e581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.341+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0046a0 tx=0x7f00cc00b970 comp rx=0 tx=0).stop 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3683892560 shutdown_connections 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e57db0 0x560d10e581c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10cd8e40 0x560d10e5d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3683892560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e57870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3683892560 >> 172.21.15.16:0/3683892560 conn(0x560d10d38fc0 msgr2=0x560d10e528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3683892560 shutdown_connections 2024-05-27T07:09:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.342+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3683892560 wait complete. 2024-05-27T07:09:09.339 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:08 smithi164 ceph-mon[35327]: pgmap v253: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 26 MiB/s wr, 497 op/s 2024-05-27T07:09:09.339 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:08 smithi164 ceph-mon[35327]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.343+0000 7f00f4ac1b80 1 -- start start 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.344+0000 7f00f4ac1b80 1 Processor -- start 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.345+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.345+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 0x560d10e61ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00df400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00df400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:59920/0 (socket says 172.21.15.16:59920) 2024-05-27T07:09:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00dea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 0x560d10e61ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00dea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 0x560d10e61ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47254/0 (socket says 172.21.15.16:47254) 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d10c7bfa0 con 0x560d10e5dd30 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d10d16760 con 0x560d10cd8e40 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.346+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d10e64800 con 0x560d10e57db0 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.347+0000 7f00f4ac1b80 10 client.0 _ll_register_callbacks cb 0x560d10d1fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.347+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.347+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32822/0 (socket says 172.21.15.16:32822) 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.347+0000 7f00dfe00700 1 -- 172.21.15.16:0/2692953588 learned_addr learned my addr 172.21.15.16:0/2692953588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00dfe00700 1 -- 172.21.15.16:0/2692953588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 msgr2=0x560d10e61ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 0x560d10e61ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00dfe00700 1 -- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 msgr2=0x560d10e61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00dfe00700 1 -- 172.21.15.16:0/2692953588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d10cc58d0 con 0x560d10e5dd30 2024-05-27T07:09:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.348+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00cc015070 con 0x560d10e5dd30 2024-05-27T07:09:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.349+0000 7f00c9600700 10 client.16284 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:09:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.349+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00cc004270 tx=0x7f00cc01aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ba57175a6f3baa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.352+0000 7f00c9600700 1 -- 172.21.15.16:0/2692953588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc005230 con 0x560d10e5dd30 2024-05-27T07:09:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.352+0000 7f00c9600700 1 -- 172.21.15.16:0/2692953588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9613+0+0 (secure 0 0 0) 0x7f00cc005030 con 0x560d10e5dd30 2024-05-27T07:09:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.352+0000 7f00c9600700 1 -- 172.21.15.16:0/2692953588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00cc005230 con 0x560d10e5dd30 2024-05-27T07:09:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.353+0000 7f00f4ac1b80 10 client.16284 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.353+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=31}) -- 0x560d10d46060 con 0x560d10e5dd30 2024-05-27T07:09:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.359+0000 7f00c9600700 1 -- 172.21.15.16:0/2692953588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f00cc0053d0 con 0x560d10e5dd30 2024-05-27T07:09:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.359+0000 7f00f4ac1b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.359+0000 7f00f4ac1b80 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:09:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.359+0000 7f00f4ac1b80 20 client.16284 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.360+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10e715b0 0x560d10e739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.360+0000 7f00f4ac1b80 4 client.16284 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:09:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.360+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d10d87d90 con 0x560d10e715b0 2024-05-27T07:09:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.360+0000 7f00df400700 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10e715b0 0x560d10e739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.361+0000 7f00df400700 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10e715b0 0x560d10e739d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=798f36ec6c82aacc server_cookie=702b5c2e1cb5ea99 in_seq=0 out_seq=0 2024-05-27T07:09:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.361+0000 7f00c9600700 10 client.16284 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:09:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.362+0000 7f00c9600700 1 -- 172.21.15.16:0/2692953588 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d10d87d90 con 0x560d10e715b0 2024-05-27T07:09:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.362+0000 7f00c9600700 10 client.16284 handle_command_reply: tid=0 2024-05-27T07:09:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.362+0000 7f00f4ac1b80 1 client.16284 shutdown 2024-05-27T07:09:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 msgr2=0x560d10e62010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00cc004270 tx=0x7f00cc01aee0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 shutdown_connections 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10e715b0 0x560d10e739d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e5dd30 0x560d10e62010 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2692953588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e57db0 0x560d10e61ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 >> 172.21.15.16:0/2692953588 conn(0x560d10d38fc0 msgr2=0x560d10e52580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00c9600700 0 client.16284 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:09:09.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.363+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 shutdown_connections 2024-05-27T07:09:09.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.365+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2692953588 wait complete. 2024-05-27T07:09:09.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.365+0000 7f00f4ac1b80 20 client.16284 trim_cache size 0 max 16384 2024-05-27T07:09:09.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:08 smithi069 ceph-mon[35276]: pgmap v253: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 26 MiB/s wr, 497 op/s 2024-05-27T07:09:09.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:08 smithi069 ceph-mon[35276]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:09:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.398+0000 7f00f4ac1b80 1 Processor -- start 2024-05-27T07:09:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 -- start start 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b6db0 con 0x560d10e57db0 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b88f0 con 0x560d10cd8e40 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.399+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b7480 con 0x560d10e5dd30 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.400+0000 7f00dea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.400+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.400+0000 7f00dfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47270/0 (socket says 172.21.15.16:47270) 2024-05-27T07:09:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.400+0000 7f00dfe00700 1 -- 172.21.15.16:0/2684772888 learned_addr learned my addr 172.21.15.16:0/2684772888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.401+0000 7f00df400700 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.402+0000 7f00dfe00700 1 -- 172.21.15.16:0/2684772888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 msgr2=0x560d10e61bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.402+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.402+0000 7f00dfe00700 1 -- 172.21.15.16:0/2684772888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 msgr2=0x560d10e5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.402+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e5f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.402+0000 7f00dfe00700 1 -- 172.21.15.16:0/2684772888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d10e54280 con 0x560d10e5dd30 2024-05-27T07:09:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.403+0000 7f00dfe00700 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00bd40 tx=0x7f00cc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfee1aced28e7ee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.404+0000 7f00cbe00700 1 -- 172.21.15.16:0/2684772888 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc004a00 con 0x560d10e5dd30 2024-05-27T07:09:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.405+0000 7f00cbe00700 1 -- 172.21.15.16:0/2684772888 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00cc015070 con 0x560d10e5dd30 2024-05-27T07:09:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.405+0000 7f00cbe00700 1 -- 172.21.15.16:0/2684772888 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc003d80 con 0x560d10e5dd30 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.406+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2684772888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 msgr2=0x560d10e5bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.406+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00bd40 tx=0x7f00cc020000 comp rx=0 tx=0).stop 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.406+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2684772888 shutdown_connections 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.406+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10e5dd30 0x560d10e5bc40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.406+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10cd8e40 0x560d10e61bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.407+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/2684772888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.407+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2684772888 >> 172.21.15.16:0/2684772888 conn(0x560d10d38fc0 msgr2=0x560d10e62a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.407+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2684772888 shutdown_connections 2024-05-27T07:09:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.409+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/2684772888 wait complete. 2024-05-27T07:09:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.410+0000 7f00f4ac1b80 1 -- start start 2024-05-27T07:09:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.410+0000 7f00f4ac1b80 1 Processor -- start 2024-05-27T07:09:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e5dd30 0x560d10e78aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b8220 con 0x560d10e57db0 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b7b50 con 0x560d10e5dd30 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b4d70 con 0x560d10cd8e40 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.411+0000 7f00f4ac1b80 10 client.0 _ll_register_callbacks cb 0x560d10d1fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.412+0000 7f00df400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:47276/0 (socket says 172.21.15.16:47276) 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 -- 172.21.15.16:0/3845958300 learned_addr learned my addr 172.21.15.16:0/3845958300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00dfe00700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e5dd30 0x560d10e78aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00dea00700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 -- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e5dd30 msgr2=0x560d10e78aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e5dd30 0x560d10e78aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 -- 172.21.15.16:0/3845958300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 msgr2=0x560d10e78560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e78560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.413+0000 7f00df400700 1 -- 172.21.15.16:0/3845958300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d10e78fe0 con 0x560d10cd8e40 2024-05-27T07:09:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.416+0000 7f00df400700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009f90 tx=0x7f00d8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e2473393dab61e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.416+0000 7f00c9600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:09:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.416+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8038060 con 0x560d10cd8e40 2024-05-27T07:09:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.417+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9613+0+0 (secure 0 0 0) 0x7f00d8007500 con 0x560d10cd8e40 2024-05-27T07:09:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.417+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00cc015070 con 0x560d10cd8e40 2024-05-27T07:09:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.417+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00d8007700 con 0x560d10cd8e40 2024-05-27T07:09:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.418+0000 7f00f4ac1b80 10 client.26086 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.418+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560d10eeca30 con 0x560d10cd8e40 2024-05-27T07:09:09.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.418+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00cc015070 con 0x560d10cd8e40 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.420+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f00d8032070 con 0x560d10cd8e40 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 10 client.26086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 20 client.26086 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10eed090 0x560d10eef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 4 client.26086 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d10d81570 con 0x560d10eed090 2024-05-27T07:09:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.421+0000 7f00dea00700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10eed090 0x560d10eef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.425+0000 7f00c9600700 10 client.26086 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:09:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.425+0000 7f00dea00700 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10eed090 0x560d10eef4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c031eeef9bb6f55d server_cookie=621eb5b493dd9091 in_seq=0 out_seq=0 2024-05-27T07:09:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.426+0000 7f00c9600700 1 -- 172.21.15.16:0/3845958300 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d10d81570 con 0x560d10eed090 2024-05-27T07:09:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.426+0000 7f00c9600700 10 client.26086 handle_command_reply: tid=0 2024-05-27T07:09:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.426+0000 7f00f4ac1b80 1 client.26086 shutdown 2024-05-27T07:09:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.428+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 msgr2=0x560d10e69ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.428+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009f90 tx=0x7f00d8009920 comp rx=0 tx=0).stop 2024-05-27T07:09:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.428+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 shutdown_connections 2024-05-27T07:09:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.428+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x560d10eed090 0x560d10eef4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d10e5dd30 0x560d10e78aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00c9600700 0 client.26086 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d10cd8e40 0x560d10e69ed0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 --2- 172.21.15.16:0/3845958300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d10e57db0 0x560d10e78560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 >> 172.21.15.16:0/3845958300 conn(0x560d10d38fc0 msgr2=0x560d10e5b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 shutdown_connections 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.429+0000 7f00f4ac1b80 1 -- 172.21.15.16:0/3845958300 wait complete. 2024-05-27T07:09:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.430+0000 7f00f4ac1b80 20 client.26086 trim_cache size 0 max 16384 2024-05-27T07:09:09.421 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 msgr2=0x7f00c40b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 0x7f00c40b5000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009da0 tx=0x7f00d80096c0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 msgr2=0x7f00e01a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f00d000e9a0 tx=0x7f00d000ecb0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 shutdown_connections 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f00c40b2af0 0x7f00c40b5000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00e0115360 0x7f00e01a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e0075b50 0x7f00e01a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 --2- 172.21.15.16:0/792014819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00764a0 0x7f00e01a1930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 >> 172.21.15.16:0/792014819 conn(0x7f00e0100030 msgr2=0x7f00e0100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 shutdown_connections 2024-05-27T07:09:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:09.431+0000 7f00e6000700 1 -- 172.21.15.16:0/792014819 wait complete. 2024-05-27T07:09:10.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:09:11.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:11 smithi164 ceph-mon[35327]: pgmap v255: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 29 MiB/s wr, 547 op/s 2024-05-27T07:09:11.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:11 smithi016 ceph-mon[31415]: pgmap v255: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 29 MiB/s wr, 547 op/s 2024-05-27T07:09:11.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:11 smithi069 ceph-mon[35276]: pgmap v255: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 29 MiB/s wr, 547 op/s 2024-05-27T07:09:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:13.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:13 smithi016 ceph-mon[31415]: pgmap v256: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 499 op/s 2024-05-27T07:09:13.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:13 smithi164 ceph-mon[35327]: pgmap v256: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 499 op/s 2024-05-27T07:09:13.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:13 smithi069 ceph-mon[35276]: pgmap v256: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 499 op/s 2024-05-27T07:09:15.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:14 smithi164 ceph-mon[35327]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:09:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:14 smithi016 ceph-mon[31415]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:09:15.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:14 smithi069 ceph-mon[35276]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:09:16.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:15 smithi164 ceph-mon[35327]: pgmap v258: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 355 op/s 2024-05-27T07:09:16.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:15 smithi016 ceph-mon[31415]: pgmap v258: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 355 op/s 2024-05-27T07:09:16.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:15 smithi069 ceph-mon[35276]: pgmap v258: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 355 op/s 2024-05-27T07:09:18.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:17 smithi016 ceph-mon[31415]: pgmap v259: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 511 op/s 2024-05-27T07:09:18.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:17 smithi164 ceph-mon[35327]: pgmap v259: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 511 op/s 2024-05-27T07:09:18.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:17 smithi069 ceph-mon[35276]: pgmap v259: 129 pgs: 129 active+clean; 18 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 511 op/s 2024-05-27T07:09:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:19.997+0000 7f67a7c00700 1 -- 172.21.15.16:0/435591091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 msgr2=0x7f67a010ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:19.997+0000 7f67a7c00700 1 --2- 172.21.15.16:0/435591091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a010ad90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f678c009990 tx=0x7f678c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:09:19.990 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:19 smithi016 ceph-mon[31415]: pgmap v260: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 543 op/s 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 -- 172.21.15.16:0/435591091 shutdown_connections 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 --2- 172.21.15.16:0/435591091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a010b2d0 0x7f67a0117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 --2- 172.21.15.16:0/435591091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a0110960 0x7f67a0110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 --2- 172.21.15.16:0/435591091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a010ad90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 -- 172.21.15.16:0/435591091 >> 172.21.15.16:0/435591091 conn(0x7f67a0077c70 msgr2=0x7f67a00780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.000+0000 7f67a7c00700 1 -- 172.21.15.16:0/435591091 shutdown_connections 2024-05-27T07:09:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.001+0000 7f67a7c00700 1 -- 172.21.15.16:0/435591091 wait complete. 2024-05-27T07:09:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.002+0000 7f67a7c00700 1 Processor -- start 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.002+0000 7f67a7c00700 1 -- start start 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a0085380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 0x7f67a007e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a011a980 con 0x7f67a010b2d0 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67a011a800 con 0x7f67a0110960 2024-05-27T07:09:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.003+0000 7f67a7c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67a011ab00 con 0x7f67a010a980 2024-05-27T07:09:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.004+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.004+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:40512/0 (socket says 172.21.15.16:40512) 2024-05-27T07:09:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.004+0000 7f67a5c00700 1 -- 172.21.15.16:0/223956382 learned_addr learned my addr 172.21.15.16:0/223956382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.004+0000 7f67a5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a0085380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.005+0000 7f679fe00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 0x7f67a007e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.006+0000 7f67a5200700 1 -- 172.21.15.16:0/223956382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 msgr2=0x7f67a007eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f67a5200700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f67a5200700 1 -- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 msgr2=0x7f67a007e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f67a5200700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 0x7f67a007e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f67a5200700 1 -- 172.21.15.16:0/223956382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a0082eb0 con 0x7f67a010a980 2024-05-27T07:09:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f67a5c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:09:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.008+0000 7f679fe00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 0x7f67a007e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:09:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.009+0000 7f67a5200700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a0085380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6794007e90 tx=0x7f679400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.011+0000 7f679d600700 1 -- 172.21.15.16:0/223956382 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679400ddd0 con 0x7f67a010a980 2024-05-27T07:09:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.011+0000 7f679d600700 1 -- 172.21.15.16:0/223956382 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6794009e20 con 0x7f67a010a980 2024-05-27T07:09:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.012+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a00831a0 con 0x7f67a010a980 2024-05-27T07:09:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.012+0000 7f679d600700 1 -- 172.21.15.16:0/223956382 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794015400 con 0x7f67a010a980 2024-05-27T07:09:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.012+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a0083730 con 0x7f67a010a980 2024-05-27T07:09:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.026+0000 7f679d600700 1 -- 172.21.15.16:0/223956382 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f67940155a0 con 0x7f67a010a980 2024-05-27T07:09:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.028+0000 7f679d600700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 0x7f67880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.029+0000 7f679d600700 1 -- 172.21.15.16:0/223956382 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f67940e3190 con 0x7f67a010a980 2024-05-27T07:09:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.030+0000 7f679fe00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 0x7f67880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.045+0000 7f679fe00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 0x7f67880b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67a0109b20 tx=0x7f678c005fb0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.050+0000 7f67b6692b80 1 Processor -- start 2024-05-27T07:09:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 -- start start 2024-05-27T07:09:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c7323037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7322fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c73222be40 con 0x55c732303d30 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c73221b9e0 con 0x55c73217ee40 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.052+0000 7f67b6692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c73221fa20 con 0x55c7322fddb0 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.053+0000 7f67a5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c7323037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.054+0000 7f679fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.054+0000 7f679fe00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33866/0 (socket says 172.21.15.16:33866) 2024-05-27T07:09:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.054+0000 7f679fe00700 1 -- 172.21.15.16:0/611089502 learned_addr learned my addr 172.21.15.16:0/611089502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.054+0000 7f67a5c00700 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7322fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.055+0000 7f679fe00700 1 -- 172.21.15.16:0/611089502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 msgr2=0x55c7322fe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.055+0000 7f679fe00700 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7322fe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.055+0000 7f679fe00700 1 -- 172.21.15.16:0/611089502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 msgr2=0x55c7323037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.056+0000 7f679fe00700 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c7323037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.056+0000 7f679fe00700 1 -- 172.21.15.16:0/611089502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7321e4420 con 0x55c732303d30 2024-05-27T07:09:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.057+0000 7f679fe00700 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f678c009990 tx=0x7f678c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cc8874ccf3b838 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.058+0000 7f6792000700 1 -- 172.21.15.16:0/611089502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c003e40 con 0x55c732303d30 2024-05-27T07:09:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.059+0000 7f6792000700 1 -- 172.21.15.16:0/611089502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f678c02a070 con 0x55c732303d30 2024-05-27T07:09:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.059+0000 7f6792000700 1 -- 172.21.15.16:0/611089502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c02f940 con 0x55c732303d30 2024-05-27T07:09:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 -- 172.21.15.16:0/611089502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 msgr2=0x55c7322fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f678c009990 tx=0x7f678c01d000 comp rx=0 tx=0).stop 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 -- 172.21.15.16:0/611089502 shutdown_connections 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7322fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c7323037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 --2- 172.21.15.16:0/611089502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732303d30 0x55c7322fd870 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.060+0000 7f67b6692b80 1 -- 172.21.15.16:0/611089502 >> 172.21.15.16:0/611089502 conn(0x55c7321defc0 msgr2=0x55c7322f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.062+0000 7f67b6692b80 1 -- 172.21.15.16:0/611089502 shutdown_connections 2024-05-27T07:09:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.062+0000 7f67b6692b80 1 -- 172.21.15.16:0/611089502 wait complete. 2024-05-27T07:09:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.063+0000 7f67b6692b80 1 -- start start 2024-05-27T07:09:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.064+0000 7f67b6692b80 1 Processor -- start 2024-05-27T07:09:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.066+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c732309e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.067+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7323070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.067+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c732307a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.067+0000 7f67b6692b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c732121fa0 con 0x55c73217ee40 2024-05-27T07:09:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.067+0000 7f67b6692b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7321bc760 con 0x55c732307620 2024-05-27T07:09:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.068+0000 7f67b6692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c73230a7a0 con 0x55c7322fddb0 2024-05-27T07:09:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.068+0000 7f67b6692b80 10 client.0 _ll_register_callbacks cb 0x55c7321c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.070+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c732307a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.070+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c732307a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:40536/0 (socket says 172.21.15.16:40536) 2024-05-27T07:09:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.070+0000 7f67a5c00700 1 -- 172.21.15.16:0/1009828191 learned_addr learned my addr 172.21.15.16:0/1009828191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.070+0000 7f67a5200700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c732309e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.070+0000 7f679fe00700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7323070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.072+0000 7f679fe00700 1 -- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 msgr2=0x55c732307a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.072+0000 7f679fe00700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c732307a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.073+0000 7f679fe00700 1 -- 172.21.15.16:0/1009828191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 msgr2=0x55c732309e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.073+0000 7f679fe00700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c732309e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.073+0000 7f679fe00700 1 -- 172.21.15.16:0/1009828191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73216b8d0 con 0x55c7322fddb0 2024-05-27T07:09:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.073+0000 7f67a5200700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c732309e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:09:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.075+0000 7f6786a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:09:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.075+0000 7f679fe00700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7323070e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f678c000c00 tx=0x7f678c00bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ce611fd75c7aad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.075+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c00be60 con 0x55c7322fddb0 2024-05-27T07:09:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.076+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f678c01e070 con 0x55c7322fddb0 2024-05-27T07:09:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.076+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c02a070 con 0x55c7322fddb0 2024-05-27T07:09:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.077+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f678c02f680 con 0x55c7322fddb0 2024-05-27T07:09:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.078+0000 7f67b6692b80 10 client.26098 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.078+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c7321ec060 con 0x55c7322fddb0 2024-05-27T07:09:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.078+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f678c02a070 con 0x55c7322fddb0 2024-05-27T07:09:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f678c00f7d0 con 0x55c7322fddb0 2024-05-27T07:09:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 10 client.26098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:09:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 20 client.26098 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c732317640 0x55c732319a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 4 client.26098 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:09:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.079+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73222dd90 con 0x55c732317640 2024-05-27T07:09:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.080+0000 7f67a5200700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c732317640 0x55c732319a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.081+0000 7f67a5200700 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c732317640 0x55c732319a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb2614dab5b13787 server_cookie=f06945c55d260454 in_seq=0 out_seq=0 2024-05-27T07:09:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.081+0000 7f6786a00700 10 client.26098 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:09:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.082+0000 7f6786a00700 1 -- 172.21.15.16:0/1009828191 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c73222dd90 con 0x55c732317640 2024-05-27T07:09:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.082+0000 7f6786a00700 10 client.26098 handle_command_reply: tid=0 2024-05-27T07:09:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.083+0000 7f67b6692b80 1 client.26098 shutdown 2024-05-27T07:09:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.083+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 msgr2=0x55c7323070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.083+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7323070e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f678c000c00 tx=0x7f678c00bc40 comp rx=0 tx=0).stop 2024-05-27T07:09:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 shutdown_connections 2024-05-27T07:09:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c732317640 0x55c732319a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f6786a00700 0 client.26098 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:09:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c732307a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c732309e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 --2- 172.21.15.16:0/1009828191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c7323070e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.084+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 >> 172.21.15.16:0/1009828191 conn(0x55c7321defc0 msgr2=0x55c7322f7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.085+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 shutdown_connections 2024-05-27T07:09:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.085+0000 7f67b6692b80 1 -- 172.21.15.16:0/1009828191 wait complete. 2024-05-27T07:09:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.086+0000 7f67b6692b80 20 client.26098 trim_cache size 0 max 16384 2024-05-27T07:09:20.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:19 smithi164 ceph-mon[35327]: pgmap v260: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 543 op/s 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.124+0000 7f67b6692b80 1 Processor -- start 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.124+0000 7f67b6692b80 1 -- start start 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c7322fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c7323037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01bf540 con 0x55c73217ee40 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01c1080 con 0x55c732307620 2024-05-27T07:09:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.125+0000 7f67b6692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01bfc10 con 0x55c7322fddb0 2024-05-27T07:09:20.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:19 smithi069 ceph-mon[35276]: pgmap v260: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 543 op/s 2024-05-27T07:09:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.129+0000 7f679fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.129+0000 7f679fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:49906/0 (socket says 172.21.15.16:49906) 2024-05-27T07:09:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.129+0000 7f679fe00700 1 -- 172.21.15.16:0/2289495312 learned_addr learned my addr 172.21.15.16:0/2289495312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.129+0000 7f67a5c00700 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c7323037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.130+0000 7f679fe00700 1 -- 172.21.15.16:0/2289495312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 msgr2=0x55c7323037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.130+0000 7f679fe00700 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c7323037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.130+0000 7f679fe00700 1 -- 172.21.15.16:0/2289495312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 msgr2=0x55c7322fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.130+0000 7f679fe00700 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c7322fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.130+0000 7f679fe00700 1 -- 172.21.15.16:0/2289495312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c732309980 con 0x55c7322fddb0 2024-05-27T07:09:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.131+0000 7f67a5c00700 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c7323037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:09:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.131+0000 7f679fe00700 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f678c00fd40 tx=0x7f678c005090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eec06d9b7bcf0fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.133+0000 7f6791600700 1 -- 172.21.15.16:0/2289495312 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c0054c0 con 0x55c7322fddb0 2024-05-27T07:09:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.133+0000 7f6791600700 1 -- 172.21.15.16:0/2289495312 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f678c02a070 con 0x55c7322fddb0 2024-05-27T07:09:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.134+0000 7f6791600700 1 -- 172.21.15.16:0/2289495312 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c02fb40 con 0x55c7322fddb0 2024-05-27T07:09:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 -- 172.21.15.16:0/2289495312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 msgr2=0x55c732309440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f678c00fd40 tx=0x7f678c005090 comp rx=0 tx=0).stop 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 -- 172.21.15.16:0/2289495312 shutdown_connections 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c732307620 0x55c7323037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73217ee40 0x55c7322fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 --2- 172.21.15.16:0/2289495312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c7322fddb0 0x55c732309440 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f678c00fd40 tx=0x7f678c005090 comp rx=0 tx=0).stop 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.136+0000 7f67b6692b80 1 -- 172.21.15.16:0/2289495312 >> 172.21.15.16:0/2289495312 conn(0x55c7321defc0 msgr2=0x55c732306ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.137+0000 7f67b6692b80 1 -- 172.21.15.16:0/2289495312 shutdown_connections 2024-05-27T07:09:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.137+0000 7f67b6692b80 1 -- 172.21.15.16:0/2289495312 wait complete. 2024-05-27T07:09:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.138+0000 7f67b6692b80 1 -- start start 2024-05-27T07:09:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 Processor -- start 2024-05-27T07:09:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 0x55c732308970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01c09b0 con 0x55c732307620 2024-05-27T07:09:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01c02e0 con 0x55c73217ee40 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01bd4e0 con 0x55c732317640 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.140+0000 7f67b6692b80 10 client.0 _ll_register_callbacks cb 0x55c7321c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.141+0000 7f67a5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.141+0000 7f67a5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:40562/0 (socket says 172.21.15.16:40562) 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.141+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.142+0000 7f67a5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:49912/0 (socket says 172.21.15.16:49912) 2024-05-27T07:09:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.142+0000 7f67a5c00700 1 -- 172.21.15.16:0/3957673958 learned_addr learned my addr 172.21.15.16:0/3957673958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.142+0000 7f67a5200700 1 -- 172.21.15.16:0/3957673958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 msgr2=0x55c732308eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.142+0000 7f67a5200700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.142+0000 7f679fe00700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 0x55c732308970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.143+0000 7f67a5200700 1 -- 172.21.15.16:0/3957673958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 msgr2=0x55c732308970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.145+0000 7f67a5200700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 0x55c732308970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.145+0000 7f67a5200700 1 -- 172.21.15.16:0/3957673958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73237fea0 con 0x55c73217ee40 2024-05-27T07:09:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.145+0000 7f67a5c00700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:09:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.145+0000 7f679fe00700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 0x55c732308970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.149+0000 7f67a5200700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67940e04f0 tx=0x7f67940aa8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3ed728f04804ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.149+0000 7f6786a00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.150+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67940b2ee0 con 0x55c73217ee40 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.150+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f67940a6670 con 0x55c73217ee40 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.150+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c02a070 con 0x55c73217ee40 2024-05-27T07:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.151+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67940c9050 con 0x55c73217ee40 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.151+0000 7f67b6692b80 10 client.36164 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.151+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c732381020 con 0x55c73217ee40 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.151+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f678c02a070 con 0x55c73217ee40 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.152+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f67940b2a50 con 0x55c73217ee40 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 10 client.36164 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 10 client.36164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 20 client.36164 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c7323816b0 0x55c732383ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 4 client.36164 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c732227570 con 0x55c7323816b0 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.153+0000 7f679fe00700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c7323816b0 0x55c732383ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.154+0000 7f679fe00700 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c7323816b0 0x55c732383ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9aadb3b9995defc server_cookie=7de57c8fd3157b02 in_seq=0 out_seq=0 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.154+0000 7f6786a00700 10 client.36164 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f6786a00700 1 -- 172.21.15.16:0/3957673958 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c732227570 con 0x55c7323816b0 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f6786a00700 10 client.36164 handle_command_reply: tid=0 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 client.36164 shutdown 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 msgr2=0x55c732308430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67940e04f0 tx=0x7f67940aa8e0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 shutdown_connections 2024-05-27T07:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55c7323816b0 0x55c732383ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c732317640 0x55c732308eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73217ee40 0x55c732308430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 --2- 172.21.15.16:0/3957673958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732307620 0x55c732308970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f6786a00700 0 client.36164 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.155+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 >> 172.21.15.16:0/3957673958 conn(0x55c7321defc0 msgr2=0x55c732306ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.156+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 shutdown_connections 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.156+0000 7f67b6692b80 1 -- 172.21.15.16:0/3957673958 wait complete. 2024-05-27T07:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.156+0000 7f67b6692b80 20 client.36164 trim_cache size 0 max 16384 2024-05-27T07:09:20.153 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:09:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.157+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 msgr2=0x7f67880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.157+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 0x7f67880b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67a0109b20 tx=0x7f678c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.157+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 msgr2=0x7f67a0085380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.157+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a0085380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6794007e90 tx=0x7f679400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 shutdown_connections 2024-05-27T07:09:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f67880b2a70 0x7f67880b4f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67a0110960 0x7f67a007eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a010a980 0x7f67a0085380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 --2- 172.21.15.16:0/223956382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a010b2d0 0x7f67a007e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.158+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 >> 172.21.15.16:0/223956382 conn(0x7f67a0077c70 msgr2=0x7f67a007b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.159+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 shutdown_connections 2024-05-27T07:09:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:20.159+0000 7f67a7c00700 1 -- 172.21.15.16:0/223956382 wait complete. 2024-05-27T07:09:21.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:09:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:21 smithi164 ceph-mon[35327]: pgmap v261: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 26 MiB/s wr, 494 op/s 2024-05-27T07:09:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:21 smithi016 ceph-mon[31415]: pgmap v261: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 26 MiB/s wr, 494 op/s 2024-05-27T07:09:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:21 smithi069 ceph-mon[35276]: pgmap v261: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 26 MiB/s wr, 494 op/s 2024-05-27T07:09:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:23 smithi164 ceph-mon[35327]: pgmap v262: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 28 MiB/s wr, 533 op/s 2024-05-27T07:09:24.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:23 smithi016 ceph-mon[31415]: pgmap v262: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 28 MiB/s wr, 533 op/s 2024-05-27T07:09:24.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:23 smithi069 ceph-mon[35276]: pgmap v262: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 28 MiB/s wr, 533 op/s 2024-05-27T07:09:26.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:25 smithi164 ceph-mon[35327]: pgmap v263: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:26.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:25 smithi016 ceph-mon[31415]: pgmap v263: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:26.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:25 smithi069 ceph-mon[35276]: pgmap v263: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:28.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:27 smithi164 ceph-mon[35327]: pgmap v264: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 568 op/s 2024-05-27T07:09:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:27 smithi016 ceph-mon[31415]: pgmap v264: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 568 op/s 2024-05-27T07:09:28.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:27 smithi069 ceph-mon[35276]: pgmap v264: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 568 op/s 2024-05-27T07:09:29.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:29 smithi016 ceph-mon[31415]: pgmap v265: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 28 MiB/s wr, 521 op/s 2024-05-27T07:09:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:29 smithi164 ceph-mon[35327]: pgmap v265: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 28 MiB/s wr, 521 op/s 2024-05-27T07:09:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:29 smithi069 ceph-mon[35276]: pgmap v265: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 28 MiB/s wr, 521 op/s 2024-05-27T07:09:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 -- 172.21.15.16:0/1734951901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc0764a0 msgr2=0x7f21bc114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 --2- 172.21.15.16:0/1734951901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc0764a0 0x7f21bc114e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b0a0 tx=0x7f21a400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 -- 172.21.15.16:0/1734951901 shutdown_connections 2024-05-27T07:09:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 --2- 172.21.15.16:0/1734951901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc115360 0x7f21bc117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 --2- 172.21.15.16:0/1734951901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21bc075b50 0x7f21bc075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 --2- 172.21.15.16:0/1734951901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc0764a0 0x7f21bc114e20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.462+0000 7f21c2400700 1 -- 172.21.15.16:0/1734951901 >> 172.21.15.16:0/1734951901 conn(0x7f21bc100030 msgr2=0x7f21bc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.463+0000 7f21c2400700 1 -- 172.21.15.16:0/1734951901 shutdown_connections 2024-05-27T07:09:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.465+0000 7f21c2400700 1 -- 172.21.15.16:0/1734951901 wait complete. 2024-05-27T07:09:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.466+0000 7f21c2400700 1 Processor -- start 2024-05-27T07:09:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.467+0000 7f21c2400700 1 -- start start 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 0x7f21bc1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21bc0764a0 0x7f21bc1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 0x7f21bc1a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc11a960 con 0x7f21bc0764a0 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc11a7e0 con 0x7f21bc115360 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21c2400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc11aae0 con 0x7f21bc075b50 2024-05-27T07:09:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.470+0000 7f21bb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 0x7f21bc1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.471+0000 7f21bb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 0x7f21bc1a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59784/0 (socket says 172.21.15.16:59784) 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.471+0000 7f21bb400700 1 -- 172.21.15.16:0/2893203256 learned_addr learned my addr 172.21.15.16:0/2893203256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.471+0000 7f21bbe00700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 0x7f21bc1a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.472+0000 7f21bbe00700 1 -- 172.21.15.16:0/2893203256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 msgr2=0x7f21bc1a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.472+0000 7f21bbe00700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 0x7f21bc1a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.472+0000 7f21bbe00700 1 -- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21bc0764a0 msgr2=0x7f21bc1a1970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.472+0000 7f21bbe00700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21bc0764a0 0x7f21bc1a1970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.472+0000 7f21bbe00700 1 -- 172.21.15.16:0/2893203256 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc1a62a0 con 0x7f21bc115360 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.473+0000 7f21bbe00700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 0x7f21bc1a5c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21ac01f980 tx=0x7f21ac01fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.474+0000 7f21b3e00700 1 -- 172.21.15.16:0/2893203256 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00cbd0 con 0x7f21bc115360 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.474+0000 7f21b3e00700 1 -- 172.21.15.16:0/2893203256 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21ac01fe70 con 0x7f21bc115360 2024-05-27T07:09:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.474+0000 7f21b3e00700 1 -- 172.21.15.16:0/2893203256 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00cbd0 con 0x7f21bc115360 2024-05-27T07:09:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.476+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc1a6590 con 0x7f21bc115360 2024-05-27T07:09:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.476+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1085f0 con 0x7f21bc115360 2024-05-27T07:09:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.488+0000 7f21b3e00700 1 -- 172.21.15.16:0/2893203256 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f21ac004d70 con 0x7f21bc115360 2024-05-27T07:09:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.490+0000 7f21b3e00700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 0x7f219c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.491+0000 7f21b3e00700 1 -- 172.21.15.16:0/2893203256 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f21ac0f3540 con 0x7f21bc115360 2024-05-27T07:09:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.492+0000 7f21bb400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 0x7f219c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.493+0000 7f21bb400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 0x7f219c0b5180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009d20 tx=0x7f21b4009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.506+0000 7f21d0ea6b80 1 Processor -- start 2024-05-27T07:09:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.508+0000 7f21d0ea6b80 1 -- start start 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.509+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.509+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027afde40 0x565027c7c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.509+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.509+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565027baae40 con 0x565027be99f0 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.509+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565027b9a9e0 con 0x565027c7ce70 2024-05-27T07:09:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565027b9ea20 con 0x565027afde40 2024-05-27T07:09:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36516/0 (socket says 172.21.15.16:36516) 2024-05-27T07:09:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 -- 172.21.15.16:0/2990413201 learned_addr learned my addr 172.21.15.16:0/2990413201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 -- 172.21.15.16:0/2990413201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027afde40 msgr2=0x565027c7c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027afde40 0x565027c7c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 -- 172.21.15.16:0/2990413201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 msgr2=0x565027c89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.510+0000 7f21bb400700 1 -- 172.21.15.16:0/2990413201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565027b63420 con 0x565027be99f0 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.511+0000 7f21bb400700 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f21b400e540 tx=0x7f21b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c41f7fa6cf949bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.512+0000 7f21b0c00700 1 -- 172.21.15.16:0/2990413201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4026430 con 0x565027be99f0 2024-05-27T07:09:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.512+0000 7f21b0c00700 1 -- 172.21.15.16:0/2990413201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b401c8a0 con 0x565027be99f0 2024-05-27T07:09:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.512+0000 7f21b0c00700 1 -- 172.21.15.16:0/2990413201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b401d7e0 con 0x565027be99f0 2024-05-27T07:09:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.513+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/2990413201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 msgr2=0x565027c7c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.513+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f21b400e540 tx=0x7f21b4014000 comp rx=0 tx=0).stop 2024-05-27T07:09:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.514+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/2990413201 shutdown_connections 2024-05-27T07:09:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.517+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.517+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027be99f0 0x565027c7c330 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.517+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/2990413201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027afde40 0x565027c7c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.517+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/2990413201 >> 172.21.15.16:0/2990413201 conn(0x565027b97390 msgr2=0x565027c784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.521+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/2990413201 shutdown_connections 2024-05-27T07:09:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.521+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/2990413201 wait complete. 2024-05-27T07:09:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.522+0000 7f21d0ea6b80 1 -- start start 2024-05-27T07:09:30.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.522+0000 7f21d0ea6b80 1 Processor -- start 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027c86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027c87110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565027aa0fa0 con 0x565027afde40 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565027b3b760 con 0x565027be99f0 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565027c89c20 con 0x565027c7ce70 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21d0ea6b80 10 client.0 _ll_register_callbacks cb 0x565027b44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.523+0000 7f21baa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027c86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21baa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027c86bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:56926/0 (socket says 172.21.15.16:56926) 2024-05-27T07:09:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36520/0 (socket says 172.21.15.16:36520) 2024-05-27T07:09:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21baa00700 1 -- 172.21.15.16:0/4232839944 learned_addr learned my addr 172.21.15.16:0/4232839944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 -- 172.21.15.16:0/4232839944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 msgr2=0x565027c87110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027c87110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 -- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 msgr2=0x565027c86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027c86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.524+0000 7f21bb400700 1 -- 172.21.15.16:0/4232839944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565027aea8d0 con 0x565027afde40 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.525+0000 7f21bb400700 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009be0 tx=0x7f21b401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae2d6da1984aa5d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.525+0000 7f21a9600700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:09:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.525+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b401cc60 con 0x565027afde40 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.525+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b401c8a0 con 0x565027afde40 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.526+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f21b4005040 con 0x565027afde40 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.526+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21b4005240 con 0x565027afde40 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.526+0000 7f21d0ea6b80 10 client.16308 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.526+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565027b6b060 con 0x565027afde40 2024-05-27T07:09:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.526+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b401c8a0 con 0x565027afde40 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f21b401d440 con 0x565027afde40 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 20 client.16308 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027ca7ad0 0x565027ca9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 4 client.16308 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:09:30.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.527+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565027bacd90 con 0x565027ca7ad0 2024-05-27T07:09:30.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.528+0000 7f21baa00700 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027ca7ad0 0x565027ca9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.532+0000 7f21baa00700 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027ca7ad0 0x565027ca9ef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1af7c35094bb95c server_cookie=bb31d02c19efbf92 in_seq=0 out_seq=0 2024-05-27T07:09:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.532+0000 7f21a9600700 10 client.16308 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.534+0000 7f21a9600700 1 -- 172.21.15.16:0/4232839944 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565027bacd90 con 0x565027ca7ad0 2024-05-27T07:09:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.534+0000 7f21a9600700 10 client.16308 handle_command_reply: tid=0 2024-05-27T07:09:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.534+0000 7f21d0ea6b80 1 client.16308 shutdown 2024-05-27T07:09:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 msgr2=0x565027c86690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009be0 tx=0x7f21b401c540 comp rx=0 tx=0).stop 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 shutdown_connections 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027ca7ad0 0x565027ca9ef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027c87110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c86690 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/4232839944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027c86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21a9600700 0 client.16308 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.535+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 >> 172.21.15.16:0/4232839944 conn(0x565027b97390 msgr2=0x565027c7c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.536+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 shutdown_connections 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.536+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/4232839944 wait complete. 2024-05-27T07:09:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.536+0000 7f21d0ea6b80 20 client.16308 trim_cache size 0 max 16384 2024-05-27T07:09:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.571+0000 7f21d0ea6b80 1 Processor -- start 2024-05-27T07:09:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.571+0000 7f21d0ea6b80 1 -- start start 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.572+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.572+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc06d9e0 con 0x565027afde40 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc06f520 con 0x565027c7ce70 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc06e0b0 con 0x565027be99f0 2024-05-27T07:09:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21baa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21baa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59808/0 (socket says 172.21.15.16:59808) 2024-05-27T07:09:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.573+0000 7f21baa00700 1 -- 172.21.15.16:0/1328997178 learned_addr learned my addr 172.21.15.16:0/1328997178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.574+0000 7f21bb400700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21baa00700 1 -- 172.21.15.16:0/1328997178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 msgr2=0x565027c7c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.574+0000 7f21bbe00700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21baa00700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c7c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21baa00700 1 -- 172.21.15.16:0/1328997178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 msgr2=0x565027c855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21baa00700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21baa00700 1 -- 172.21.15.16:0/1328997178 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565027c79000 con 0x565027be99f0 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.575+0000 7f21bbe00700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.576+0000 7f21bb400700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:09:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.576+0000 7f21baa00700 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b0a0 tx=0x7f21a4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3b7995e46a3a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.580+0000 7f21abe00700 1 -- 172.21.15.16:0/1328997178 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a400bab0 con 0x565027be99f0 2024-05-27T07:09:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.580+0000 7f21abe00700 1 -- 172.21.15.16:0/1328997178 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21a400bc50 con 0x565027be99f0 2024-05-27T07:09:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.580+0000 7f21abe00700 1 -- 172.21.15.16:0/1328997178 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4007ca0 con 0x565027be99f0 2024-05-27T07:09:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/1328997178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 msgr2=0x565027c81e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b0a0 tx=0x7f21a4009840 comp rx=0 tx=0).stop 2024-05-27T07:09:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/1328997178 shutdown_connections 2024-05-27T07:09:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027c7ce70 0x565027c7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027c855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/1328997178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027be99f0 0x565027c81e90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.582+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/1328997178 >> 172.21.15.16:0/1328997178 conn(0x565027b97390 msgr2=0x565027c7c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.583+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/1328997178 shutdown_connections 2024-05-27T07:09:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.584+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/1328997178 wait complete. 2024-05-27T07:09:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.585+0000 7f21d0ea6b80 1 -- start start 2024-05-27T07:09:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.585+0000 7f21d0ea6b80 1 Processor -- start 2024-05-27T07:09:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.590+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027cad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.590+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027caa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.591+0000 7f21d0ea6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.591+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc06ee50 con 0x565027afde40 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.591+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc06e780 con 0x565027be99f0 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.591+0000 7f21d0ea6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10f550 con 0x565027c7ce70 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.591+0000 7f21d0ea6b80 10 client.0 _ll_register_callbacks cb 0x565027b44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.593+0000 7f21bbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.593+0000 7f21bbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59818/0 (socket says 172.21.15.16:59818) 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.593+0000 7f21bbe00700 1 -- 172.21.15.16:0/789778526 learned_addr learned my addr 172.21.15.16:0/789778526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.593+0000 7f21baa00700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027caa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bbe00700 1 -- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 msgr2=0x565027caa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bb400700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027cad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bbe00700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027caa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bbe00700 1 -- 172.21.15.16:0/789778526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 msgr2=0x565027cad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bbe00700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027cad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21bbe00700 1 -- 172.21.15.16:0/789778526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565027cab240 con 0x565027c7ce70 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.594+0000 7f21baa00700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027caa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.595+0000 7f21bb400700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027cad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.595+0000 7f21bbe00700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0f0ea0 tx=0x7f21ac0ba080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546a9b9a74726759 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.595+0000 7f21a9600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:09:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.597+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0d3070 con 0x565027c7ce70 2024-05-27T07:09:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.597+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f21ac0c3010 con 0x565027c7ce70 2024-05-27T07:09:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.598+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a400bc50 con 0x565027c7ce70 2024-05-27T07:09:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.599+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21ac0c3290 con 0x565027c7ce70 2024-05-27T07:09:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.599+0000 7f21d0ea6b80 10 client.26116 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.600+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565027d11fc0 con 0x565027c7ce70 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21a400bc50 con 0x565027c7ce70 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f21ac0b4b50 con 0x565027c7ce70 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 20 client.26116 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027d12640 0x565027d14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 4 client.26116 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:09:30.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.601+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565027ba6570 con 0x565027d12640 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.602+0000 7f21bb400700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027d12640 0x565027d14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.603+0000 7f21bb400700 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027d12640 0x565027d14a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60abb00fccf4f64 server_cookie=f537893a4d4a6814 in_seq=0 out_seq=0 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.603+0000 7f21a9600700 10 client.26116 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.604+0000 7f21a9600700 1 -- 172.21.15.16:0/789778526 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565027ba6570 con 0x565027d12640 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.604+0000 7f21a9600700 10 client.26116 handle_command_reply: tid=0 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.604+0000 7f21d0ea6b80 1 client.26116 shutdown 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.604+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 msgr2=0x565027caacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.604+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0f0ea0 tx=0x7f21ac0ba080 comp rx=0 tx=0).stop 2024-05-27T07:09:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 shutdown_connections 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x565027d12640 0x565027d14a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565027c7ce70 0x565027caacd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565027afde40 0x565027cad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 --2- 172.21.15.16:0/789778526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565027be99f0 0x565027caa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 >> 172.21.15.16:0/789778526 conn(0x565027b97390 msgr2=0x565027c91f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.605+0000 7f21a9600700 0 client.26116 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.606+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 shutdown_connections 2024-05-27T07:09:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.606+0000 7f21d0ea6b80 1 -- 172.21.15.16:0/789778526 wait complete. 2024-05-27T07:09:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.606+0000 7f21d0ea6b80 20 client.26116 trim_cache size 0 max 16384 2024-05-27T07:09:30.599 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:09:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 msgr2=0x7f219c0b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 0x7f219c0b5180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009d20 tx=0x7f21b4009260 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 msgr2=0x7f21bc1a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 0x7f21bc1a5c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21ac01f980 tx=0x7f21ac01fc90 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 shutdown_connections 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f219c0b2c70 0x7f219c0b5180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc115360 0x7f21bc1a5c00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21bc075b50 0x7f21bc1a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 --2- 172.21.15.16:0/2893203256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21bc0764a0 0x7f21bc1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 >> 172.21.15.16:0/2893203256 conn(0x7f21bc100030 msgr2=0x7f21bc100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.608+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 shutdown_connections 2024-05-27T07:09:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:30.609+0000 7f21c2400700 1 -- 172.21.15.16:0/2893203256 wait complete. 2024-05-27T07:09:31.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:31 smithi016 ceph-mon[31415]: pgmap v266: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 450 op/s 2024-05-27T07:09:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:31 smithi164 ceph-mon[35327]: pgmap v266: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 450 op/s 2024-05-27T07:09:32.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:31 smithi069 ceph-mon[35276]: pgmap v266: 129 pgs: 129 active+clean; 18 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 450 op/s 2024-05-27T07:09:32.871 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:09:32.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:09:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:33 smithi164 ceph-mon[35327]: pgmap v267: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 36 MiB/s wr, 678 op/s 2024-05-27T07:09:34.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:33 smithi016 ceph-mon[31415]: pgmap v267: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 36 MiB/s wr, 678 op/s 2024-05-27T07:09:34.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:33 smithi069 ceph-mon[35276]: pgmap v267: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 36 MiB/s wr, 678 op/s 2024-05-27T07:09:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:35 smithi164 ceph-mon[35327]: pgmap v268: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 24 MiB/s wr, 452 op/s 2024-05-27T07:09:36.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:35 smithi016 ceph-mon[31415]: pgmap v268: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 24 MiB/s wr, 452 op/s 2024-05-27T07:09:36.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:35 smithi069 ceph-mon[35276]: pgmap v268: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 24 MiB/s wr, 452 op/s 2024-05-27T07:09:38.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:37 smithi164 ceph-mon[35327]: pgmap v269: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 571 op/s 2024-05-27T07:09:38.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:37 smithi016 ceph-mon[31415]: pgmap v269: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 571 op/s 2024-05-27T07:09:38.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:37 smithi069 ceph-mon[35276]: pgmap v269: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 30 MiB/s wr, 571 op/s 2024-05-27T07:09:38.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:38 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:09:39.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:38 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:09:39.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:38 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:09:40.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:39 smithi164 ceph-mon[35327]: pgmap v270: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 28 MiB/s wr, 537 op/s 2024-05-27T07:09:40.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:39 smithi016 ceph-mon[31415]: pgmap v270: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 28 MiB/s wr, 537 op/s 2024-05-27T07:09:40.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:39 smithi069 ceph-mon[35276]: pgmap v270: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 28 MiB/s wr, 537 op/s 2024-05-27T07:09:41.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:41 smithi016 ceph-mon[31415]: pgmap v271: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 436 op/s 2024-05-27T07:09:41.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:41 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:09:41.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:41 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:09:41.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:41 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:09:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:41 smithi164 ceph-mon[35327]: pgmap v271: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 436 op/s 2024-05-27T07:09:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:41 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:09:42.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:41 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:09:42.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:41 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:09:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:41 smithi069 ceph-mon[35276]: pgmap v271: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 436 op/s 2024-05-27T07:09:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:41 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:09:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:41 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:09:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:41 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:09:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.853+0000 7fd5dbc00700 1 -- 172.21.15.16:0/876521657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 msgr2=0x7fd5d4110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.853+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/876521657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d4110da0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009f90 tx=0x7fd5cc00b330 comp rx=0 tx=0).stop 2024-05-27T07:09:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 -- 172.21.15.16:0/876521657 shutdown_connections 2024-05-27T07:09:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/876521657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d4110da0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/876521657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105450 0x7fd5d4105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/876521657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105da0 0x7fd5d410e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 -- 172.21.15.16:0/876521657 >> 172.21.15.16:0/876521657 conn(0x7fd5d40fedb0 msgr2=0x7fd5d41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 -- 172.21.15.16:0/876521657 shutdown_connections 2024-05-27T07:09:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.855+0000 7fd5dbc00700 1 -- 172.21.15.16:0/876521657 wait complete. 2024-05-27T07:09:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.857+0000 7fd5dbc00700 1 Processor -- start 2024-05-27T07:09:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.857+0000 7fd5dbc00700 1 -- start start 2024-05-27T07:09:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105450 0x7fd5d410b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 0x7fd5d410bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4116590 con 0x7fd5d4105da0 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4116410 con 0x7fd5d4105450 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.858+0000 7fd5dbc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4116710 con 0x7fd5d410e8c0 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.859+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105450 0x7fd5d410b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.859+0000 7fd5d9c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.859+0000 7fd5d9c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48852/0 (socket says 172.21.15.16:48852) 2024-05-27T07:09:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.859+0000 7fd5d9c00700 1 -- 172.21.15.16:0/3777793419 learned_addr learned my addr 172.21.15.16:0/3777793419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.860+0000 7fd5d9c00700 1 -- 172.21.15.16:0/3777793419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105450 msgr2=0x7fd5d410b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.860+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105450 0x7fd5d410b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.860+0000 7fd5d9c00700 1 -- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 msgr2=0x7fd5d410bc30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.862+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 0x7fd5d410bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.862+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 0x7fd5d410bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.862+0000 7fd5d9c00700 1 -- 172.21.15.16:0/3777793419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41a2ba0 con 0x7fd5d410e8c0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.862+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 0x7fd5d410bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.863+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b6c0 tx=0x7fd5cc00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.863+0000 7fd5d1600700 1 -- 172.21.15.16:0/3777793419 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc021070 con 0x7fd5d410e8c0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.864+0000 7fd5d1600700 1 -- 172.21.15.16:0/3777793419 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5cc007ac0 con 0x7fd5d410e8c0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.864+0000 7fd5d1600700 1 -- 172.21.15.16:0/3777793419 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0052a0 con 0x7fd5d410e8c0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.864+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a2e30 con 0x7fd5d410e8c0 2024-05-27T07:09:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.865+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a3320 con 0x7fd5d410e8c0 2024-05-27T07:09:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.871+0000 7fd5d1600700 1 -- 172.21.15.16:0/3777793419 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd5cc024070 con 0x7fd5d410e8c0 2024-05-27T07:09:42.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.874+0000 7fd5d1600700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 0x7fd5bc0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.874+0000 7fd5d1600700 1 -- 172.21.15.16:0/3777793419 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fd5cc00f020 con 0x7fd5d410e8c0 2024-05-27T07:09:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.876+0000 7fd5d9200700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 0x7fd5bc0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.878+0000 7fd5d9200700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 0x7fd5bc0b50d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005fd0 tx=0x7fd5c8009310 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.896+0000 7fd5ea69db80 1 Processor -- start 2024-05-27T07:09:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.896+0000 7fd5ea69db80 1 -- start start 2024-05-27T07:09:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4be58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4be6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd4b13ec0 con 0x555fd4bebdb0 2024-05-27T07:09:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555fd4b03a60 con 0x555fd4a66ec0 2024-05-27T07:09:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.897+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555fd4b07aa0 con 0x555fd4be5e30 2024-05-27T07:09:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:57626/0 (socket says 172.21.15.16:57626) 2024-05-27T07:09:42.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 -- 172.21.15.16:0/1878996278 learned_addr learned my addr 172.21.15.16:0/1878996278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4be6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 -- 172.21.15.16:0/1878996278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 msgr2=0x555fd4be6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4be6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.898+0000 7fd5d9200700 1 -- 172.21.15.16:0/1878996278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 msgr2=0x555fd4be58f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:09:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.901+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4be58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.904+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4be58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.904+0000 7fd5d9200700 1 -- 172.21.15.16:0/1878996278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd4acc4a0 con 0x555fd4a66ec0 2024-05-27T07:09:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.904+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4be58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.904+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80041d0 tx=0x7fd5c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa3c6ba140a5a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.905+0000 7fd5c6000700 1 -- 172.21.15.16:0/1878996278 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8009e60 con 0x555fd4a66ec0 2024-05-27T07:09:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.905+0000 7fd5c6000700 1 -- 172.21.15.16:0/1878996278 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c8003d10 con 0x555fd4a66ec0 2024-05-27T07:09:42.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.905+0000 7fd5c6000700 1 -- 172.21.15.16:0/1878996278 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8009e60 con 0x555fd4a66ec0 2024-05-27T07:09:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.907+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1878996278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 msgr2=0x555fd4beb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.907+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80041d0 tx=0x7fd5c8014000 comp rx=0 tx=0).stop 2024-05-27T07:09:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.908+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1878996278 shutdown_connections 2024-05-27T07:09:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.908+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4be6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.908+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4beb870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.908+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1878996278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4be58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.908+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1878996278 >> 172.21.15.16:0/1878996278 conn(0x555fd4ac7040 msgr2=0x555fd4be0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.909+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1878996278 shutdown_connections 2024-05-27T07:09:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.909+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1878996278 wait complete. 2024-05-27T07:09:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.910+0000 7fd5ea69db80 1 -- start start 2024-05-27T07:09:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.910+0000 7fd5ea69db80 1 Processor -- start 2024-05-27T07:09:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4be5e30 0x555fd4befaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4bf0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd4a0a020 con 0x555fd4bebdb0 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555fd4aa47e0 con 0x555fd4be5e30 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555fd4bf2880 con 0x555fd4a66ec0 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.911+0000 7fd5ea69db80 10 client.0 _ll_register_callbacks cb 0x555fd4aadcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.912+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.912+0000 7fd5d3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4be5e30 0x555fd4befaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.912+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:48872/0 (socket says 172.21.15.16:48872) 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.912+0000 7fd5d9200700 1 -- 172.21.15.16:0/1090548665 learned_addr learned my addr 172.21.15.16:0/1090548665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.912+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4bf0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.913+0000 7fd5d9200700 1 -- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4be5e30 msgr2=0x555fd4befaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.913+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4be5e30 0x555fd4befaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.913+0000 7fd5d9200700 1 -- 172.21.15.16:0/1090548665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 msgr2=0x555fd4bf0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.913+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4bf0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.913+0000 7fd5d9200700 1 -- 172.21.15.16:0/1090548665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd4a53950 con 0x555fd4a66ec0 2024-05-27T07:09:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.914+0000 7fd5baa00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:09:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.914+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8004970 tx=0x7fd5c80049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0c963780842fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.914+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8004dd0 con 0x555fd4a66ec0 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.915+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fd5c802d070 con 0x555fd4a66ec0 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.915+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c8003d10 con 0x555fd4a66ec0 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.915+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5c8009e60 con 0x555fd4a66ec0 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.915+0000 7fd5ea69db80 10 client.26128 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.915+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555fd4ad40e0 con 0x555fd4a66ec0 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c8004dd0 con 0x555fd4a66ec0 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd5c8031b40 con 0x555fd4a66ec0 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5ea69db80 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5ea69db80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5ea69db80 20 client.26128 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4bff600 0x555fd4c01a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.918+0000 7fd5ea69db80 4 client.26128 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:09:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.919+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd5c8003d10 con 0x555fd4bff600 2024-05-27T07:09:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.919+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4bff600 0x555fd4c01a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.920+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4bff600 0x555fd4c01a20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f399434dd7e54c7 server_cookie=253deb997379229 in_seq=0 out_seq=0 2024-05-27T07:09:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.920+0000 7fd5baa00700 10 client.26128 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.921+0000 7fd5baa00700 1 -- 172.21.15.16:0/1090548665 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fd5c8003d10 con 0x555fd4bff600 2024-05-27T07:09:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.921+0000 7fd5baa00700 10 client.26128 handle_command_reply: tid=0 2024-05-27T07:09:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.921+0000 7fd5ea69db80 1 client.26128 shutdown 2024-05-27T07:09:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 msgr2=0x555fd4bef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8004970 tx=0x7fd5c80049a0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 shutdown_connections 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4bff600 0x555fd4c01a20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4bebdb0 0x555fd4bf0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4a66ec0 0x555fd4bef5b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1090548665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4be5e30 0x555fd4befaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5baa00700 0 client.26128 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.922+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 >> 172.21.15.16:0/1090548665 conn(0x555fd4ac7040 msgr2=0x555fd4be0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.923+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 shutdown_connections 2024-05-27T07:09:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.923+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1090548665 wait complete. 2024-05-27T07:09:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.924+0000 7fd5ea69db80 20 client.26128 trim_cache size 0 max 16384 2024-05-27T07:09:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.972+0000 7fd5ea69db80 1 Processor -- start 2024-05-27T07:09:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.972+0000 7fd5ea69db80 1 -- start start 2024-05-27T07:09:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4be5e30 0x555fd4bfbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4bebdb0 0x555fd4be02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41bf400 con 0x555fd4be5e30 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41c0f40 con 0x555fd4a66ec0 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.973+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41bfad0 con 0x555fd4bebdb0 2024-05-27T07:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.974+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.974+0000 7fd5d9200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:57648/0 (socket says 172.21.15.16:57648) 2024-05-27T07:09:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.974+0000 7fd5d9200700 1 -- 172.21.15.16:0/1783775380 learned_addr learned my addr 172.21.15.16:0/1783775380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d3e00700 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4be5e30 0x555fd4bfbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d9200700 1 -- 172.21.15.16:0/1783775380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4bebdb0 msgr2=0x555fd4be02e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:09:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4bebdb0 0x555fd4be02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d9200700 1 -- 172.21.15.16:0/1783775380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4be5e30 msgr2=0x555fd4bfbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4be5e30 0x555fd4bfbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.975+0000 7fd5d9200700 1 -- 172.21.15.16:0/1783775380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd4be6ab0 con 0x555fd4a66ec0 2024-05-27T07:09:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.977+0000 7fd5d9200700 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800fed0 tx=0x7fd5c800fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3ff85332929a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.979+0000 7fd5c5600700 1 -- 172.21.15.16:0/1783775380 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8041070 con 0x555fd4a66ec0 2024-05-27T07:09:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.979+0000 7fd5c5600700 1 -- 172.21.15.16:0/1783775380 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c8004a10 con 0x555fd4a66ec0 2024-05-27T07:09:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.979+0000 7fd5c5600700 1 -- 172.21.15.16:0/1783775380 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8031e60 con 0x555fd4a66ec0 2024-05-27T07:09:42.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.980+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1783775380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 msgr2=0x555fd4bed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800fed0 tx=0x7fd5c800fce0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1783775380 shutdown_connections 2024-05-27T07:09:42.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4bebdb0 0x555fd4be02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4a66ec0 0x555fd4bed1d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/1783775380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4be5e30 0x555fd4bfbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1783775380 >> 172.21.15.16:0/1783775380 conn(0x555fd4ac7040 msgr2=0x555fd4bdfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.981+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1783775380 shutdown_connections 2024-05-27T07:09:42.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.982+0000 7fd5ea69db80 1 -- 172.21.15.16:0/1783775380 wait complete. 2024-05-27T07:09:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.982+0000 7fd5ea69db80 1 -- start start 2024-05-27T07:09:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.984+0000 7fd5ea69db80 1 Processor -- start 2024-05-27T07:09:42.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 0x555fd4c64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4c64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41c0870 con 0x555fd4a66ec0 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41c01a0 con 0x555fd4bebdb0 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41bd380 con 0x555fd4be5e30 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.986+0000 7fd5ea69db80 10 client.0 _ll_register_callbacks cb 0x555fd4aadcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.987+0000 7fd5d9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.987+0000 7fd5d9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:57660/0 (socket says 172.21.15.16:57660) 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.987+0000 7fd5d9c00700 1 -- 172.21.15.16:0/322108204 learned_addr learned my addr 172.21.15.16:0/322108204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.987+0000 7fd5d9200700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 0x555fd4c64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.987+0000 7fd5d9c00700 1 -- 172.21.15.16:0/322108204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 msgr2=0x555fd4c64890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.988+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4c64890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.988+0000 7fd5d9c00700 1 -- 172.21.15.16:0/322108204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 msgr2=0x555fd4c64350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.990+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 0x555fd4c64350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.990+0000 7fd5d9c00700 1 -- 172.21.15.16:0/322108204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd4c046b0 con 0x555fd4bebdb0 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.990+0000 7fd5d9c00700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006060 tx=0x7fd5cc0ab2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206540a4ad598194 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.990+0000 7fd5baa00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:09:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.990+0000 7fd5d9200700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 0x555fd4c64350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.991+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0c2020 con 0x555fd4bebdb0 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.991+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fd5cc0c5310 con 0x555fd4bebdb0 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.991+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c8004a10 con 0x555fd4bebdb0 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.992+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc0ab410 con 0x555fd4bebdb0 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.992+0000 7fd5ea69db80 10 client.36194 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.992+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555fd4c6b9e0 con 0x555fd4bebdb0 2024-05-27T07:09:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.993+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c8004a10 con 0x555fd4bebdb0 2024-05-27T07:09:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd5cc011070 con 0x555fd4bebdb0 2024-05-27T07:09:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 10 client.36194 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 10 client.36194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 20 client.36194 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4c6bff0 0x555fd4c6e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 4 client.36194 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:09:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.994+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555fd4b15e10 con 0x555fd4c6bff0 2024-05-27T07:09:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.995+0000 7fd5d9200700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4c6bff0 0x555fd4c6e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.995+0000 7fd5baa00700 10 client.36194 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:42.997+0000 7fd5d9200700 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4c6bff0 0x555fd4c6e450 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb382006d69fa441 server_cookie=b7501c22fb9157b in_seq=0 out_seq=0 2024-05-27T07:09:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.000+0000 7fd5baa00700 1 -- 172.21.15.16:0/322108204 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fd4b15e10 con 0x555fd4c6bff0 2024-05-27T07:09:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.000+0000 7fd5baa00700 10 client.36194 handle_command_reply: tid=0 2024-05-27T07:09:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.000+0000 7fd5ea69db80 1 client.36194 shutdown 2024-05-27T07:09:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 msgr2=0x555fd4c04070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006060 tx=0x7fd5cc0ab2a0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 shutdown_connections 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x555fd4c6bff0 0x555fd4c6e450 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fd4bebdb0 0x555fd4c04070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd4a66ec0 0x555fd4c64350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 --2- 172.21.15.16:0/322108204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fd4be5e30 0x555fd4c64890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5baa00700 0 client.36194 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.001+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 >> 172.21.15.16:0/322108204 conn(0x555fd4ac7040 msgr2=0x555fd4be0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.004+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 shutdown_connections 2024-05-27T07:09:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.004+0000 7fd5ea69db80 1 -- 172.21.15.16:0/322108204 wait complete. 2024-05-27T07:09:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.005+0000 7fd5ea69db80 20 client.36194 trim_cache size 0 max 16384 2024-05-27T07:09:42.997 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:09:42.997 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:09:42.997 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "546ce721-2ce1-4fdd-bd77-7708518d7287", 2024-05-27T07:09:42.997 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:09:42.997 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:09:42.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 msgr2=0x7fd5bc0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 0x7fd5bc0b50d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005fd0 tx=0x7fd5c8009310 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 msgr2=0x7fd5d410c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b6c0 tx=0x7fd5cc00cd50 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 shutdown_connections 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd5bc0b2bc0 0x7fd5bc0b50d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d410e8c0 0x7fd5d410c170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d4105450 0x7fd5d410b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 --2- 172.21.15.16:0/3777793419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5d4105da0 0x7fd5d410bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.007+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 >> 172.21.15.16:0/3777793419 conn(0x7fd5d40fedb0 msgr2=0x7fd5d40ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:42.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.008+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 shutdown_connections 2024-05-27T07:09:43.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:43.008+0000 7fd5dbc00700 1 -- 172.21.15.16:0/3777793419 wait complete. 2024-05-27T07:09:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:43 smithi164 ceph-mon[35327]: pgmap v272: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 34 MiB/s wr, 648 op/s 2024-05-27T07:09:44.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:43 smithi016 ceph-mon[31415]: pgmap v272: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 34 MiB/s wr, 648 op/s 2024-05-27T07:09:44.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:43 smithi069 ceph-mon[35276]: pgmap v272: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 34 MiB/s wr, 648 op/s 2024-05-27T07:09:44.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:09:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:44 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:09:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:44 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:44 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:09:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:44 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:44 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:09:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:44 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:09:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:44 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:44 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:09:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:44 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:44 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:09:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:44 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:09:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:44 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:44 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:09:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:44 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:09:45.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:44 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:09:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:45 smithi164 ceph-mon[35327]: pgmap v273: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 420 op/s 2024-05-27T07:09:46.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:45 smithi016 ceph-mon[31415]: pgmap v273: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 420 op/s 2024-05-27T07:09:46.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:45 smithi069 ceph-mon[35276]: pgmap v273: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 420 op/s 2024-05-27T07:09:48.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:47 smithi016 ceph-mon[31415]: pgmap v274: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 540 op/s 2024-05-27T07:09:48.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:47 smithi069 ceph-mon[35276]: pgmap v274: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 540 op/s 2024-05-27T07:09:48.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:47 smithi164 ceph-mon[35327]: pgmap v274: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 540 op/s 2024-05-27T07:09:50.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:49 smithi016 ceph-mon[31415]: pgmap v275: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:50.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:49 smithi069 ceph-mon[35276]: pgmap v275: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:50.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:49 smithi164 ceph-mon[35327]: pgmap v275: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 27 MiB/s wr, 518 op/s 2024-05-27T07:09:52.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:51 smithi069 ceph-mon[35276]: pgmap v276: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 428 op/s 2024-05-27T07:09:52.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:51 smithi016 ceph-mon[31415]: pgmap v276: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 428 op/s 2024-05-27T07:09:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:51 smithi164 ceph-mon[35327]: pgmap v276: 129 pgs: 129 active+clean; 19 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 428 op/s 2024-05-27T07:09:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.067+0000 7f97df200700 1 -- 172.21.15.16:0/4009011424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 msgr2=0x7f97d8113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.067+0000 7f97df200700 1 --2- 172.21.15.16:0/4009011424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8113340 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97c80098f0 tx=0x7f97c8009c00 comp rx=0 tx=0).stop 2024-05-27T07:09:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 -- 172.21.15.16:0/4009011424 shutdown_connections 2024-05-27T07:09:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 --2- 172.21.15.16:0/4009011424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8113340 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 --2- 172.21.15.16:0/4009011424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d810c5a0 0x7f97d810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 --2- 172.21.15.16:0/4009011424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97d8106560 0x7f97d8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 -- 172.21.15.16:0/4009011424 >> 172.21.15.16:0/4009011424 conn(0x7f97d8102040 msgr2=0x7f97d8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.068+0000 7f97df200700 1 -- 172.21.15.16:0/4009011424 shutdown_connections 2024-05-27T07:09:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.069+0000 7f97df200700 1 -- 172.21.15.16:0/4009011424 wait complete. 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.073+0000 7f97df200700 1 Processor -- start 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.073+0000 7f97df200700 1 -- start start 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d8106560 0x7f97d8077cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97d810c5a0 0x7f97d8078740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81165d0 con 0x7f97d8106560 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8116450 con 0x7f97d8106eb0 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.074+0000 7f97df200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8116750 con 0x7f97d810c5a0 2024-05-27T07:09:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.075+0000 7f97d7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.075+0000 7f97d7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51620/0 (socket says 172.21.15.16:51620) 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.075+0000 7f97d7400700 1 -- 172.21.15.16:0/866360243 learned_addr learned my addr 172.21.15.16:0/866360243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.076+0000 7f97d7e00700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d8106560 0x7f97d8077cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.077+0000 7f97d7400700 1 -- 172.21.15.16:0/866360243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97d810c5a0 msgr2=0x7f97d8078740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.077+0000 7f97d7400700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97d810c5a0 0x7f97d8078740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.077+0000 7f97d7400700 1 -- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d8106560 msgr2=0x7f97d8077cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.077+0000 7f97d7400700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d8106560 0x7f97d8077cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.077+0000 7f97d7400700 1 -- 172.21.15.16:0/866360243 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d819cd00 con 0x7f97d8106eb0 2024-05-27T07:09:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.081+0000 7f97d7400700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007ee0 tx=0x7f97d000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.082+0000 7f97d4c00700 1 -- 172.21.15.16:0/866360243 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d001a070 con 0x7f97d8106eb0 2024-05-27T07:09:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.082+0000 7f97d4c00700 1 -- 172.21.15.16:0/866360243 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d0015890 con 0x7f97d8106eb0 2024-05-27T07:09:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.082+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d819cff0 con 0x7f97d8106eb0 2024-05-27T07:09:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.082+0000 7f97d4c00700 1 -- 172.21.15.16:0/866360243 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0012710 con 0x7f97d8106eb0 2024-05-27T07:09:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.083+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d819d580 con 0x7f97d8106eb0 2024-05-27T07:09:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.089+0000 7f97d4c00700 1 -- 172.21.15.16:0/866360243 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f97d001d070 con 0x7f97d8106eb0 2024-05-27T07:09:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.092+0000 7f97d4c00700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 0x7f97bc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.092+0000 7f97d7e00700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 0x7f97bc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.093+0000 7f97d4c00700 1 -- 172.21.15.16:0/866360243 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f97d00e2330 con 0x7f97d8106eb0 2024-05-27T07:09:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.094+0000 7f97d7e00700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 0x7f97bc0b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97c4005950 tx=0x7f97c40058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.121+0000 7f97edce4b80 1 Processor -- start 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 -- start start 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc8d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbc8d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ffbbbbe40 con 0x562ffbb0ee40 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.122+0000 7f97edce4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ffbbab9e0 con 0x562ffbc8de70 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.123+0000 7f97edce4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ffbbafa20 con 0x562ffbbfa9f0 2024-05-27T07:09:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.123+0000 7f97dd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.124+0000 7f97dd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51626/0 (socket says 172.21.15.16:51626) 2024-05-27T07:09:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.124+0000 7f97dd200700 1 -- 172.21.15.16:0/2046343014 learned_addr learned my addr 172.21.15.16:0/2046343014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.124+0000 7f97d7400700 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbc8d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.125+0000 7f97dd200700 1 -- 172.21.15.16:0/2046343014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 msgr2=0x562ffbc8d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.125+0000 7f97dd200700 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc8d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.125+0000 7f97dd200700 1 -- 172.21.15.16:0/2046343014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 msgr2=0x562ffbc8d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.125+0000 7f97dd200700 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbc8d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.125+0000 7f97dd200700 1 -- 172.21.15.16:0/2046343014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffbb74420 con 0x562ffbc8de70 2024-05-27T07:09:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.127+0000 7f97dd200700 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009590 tx=0x7f97c80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c78b8e1a1828d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.128+0000 7f97cd600700 1 -- 172.21.15.16:0/2046343014 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8025070 con 0x562ffbc8de70 2024-05-27T07:09:53.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.128+0000 7f97cd600700 1 -- 172.21.15.16:0/2046343014 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c800f8f0 con 0x562ffbc8de70 2024-05-27T07:09:53.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.128+0000 7f97cd600700 1 -- 172.21.15.16:0/2046343014 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8003680 con 0x562ffbc8de70 2024-05-27T07:09:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.131+0000 7f97edce4b80 1 -- 172.21.15.16:0/2046343014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 msgr2=0x562ffbc9a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.132+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009590 tx=0x7f97c80095c0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.132+0000 7f97edce4b80 1 -- 172.21.15.16:0/2046343014 shutdown_connections 2024-05-27T07:09:53.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.132+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc9a110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.133+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.133+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2046343014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbc8d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.133+0000 7f97edce4b80 1 -- 172.21.15.16:0/2046343014 >> 172.21.15.16:0/2046343014 conn(0x562ffbba8390 msgr2=0x562ffbc894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.133+0000 7f97edce4b80 1 -- 172.21.15.16:0/2046343014 shutdown_connections 2024-05-27T07:09:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.134+0000 7f97edce4b80 1 -- 172.21.15.16:0/2046343014 wait complete. 2024-05-27T07:09:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.134+0000 7f97edce4b80 1 -- start start 2024-05-27T07:09:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.135+0000 7f97edce4b80 1 Processor -- start 2024-05-27T07:09:53.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.137+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.138+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbbfa9f0 0x562ffbc97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.138+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.138+0000 7f97d7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.138+0000 7f97d7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36812/0 (socket says 172.21.15.16:36812) 2024-05-27T07:09:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97edce4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ffbab1fa0 con 0x562ffbbfa9f0 2024-05-27T07:09:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97edce4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ffbb4c760 con 0x562ffbc8de70 2024-05-27T07:09:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97edce4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ffbc9ab20 con 0x562ffbb0ee40 2024-05-27T07:09:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97edce4b80 10 client.0 _ll_register_callbacks cb 0x562ffbb55c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97dd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97dd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51642/0 (socket says 172.21.15.16:51642) 2024-05-27T07:09:53.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.139+0000 7f97dd200700 1 -- 172.21.15.16:0/1796479339 learned_addr learned my addr 172.21.15.16:0/1796479339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:53.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.140+0000 7f97d7400700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbbfa9f0 0x562ffbc97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 -- 172.21.15.16:0/1796479339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 msgr2=0x562ffbc97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 -- 172.21.15.16:0/1796479339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbbfa9f0 msgr2=0x562ffbc97bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbbfa9f0 0x562ffbc97bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 -- 172.21.15.16:0/1796479339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffbafb8d0 con 0x562ffbc8de70 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97d7e00700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97dd200700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009810 tx=0x7f97c800f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dad19ec886671d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c800f8f0 con 0x562ffbc8de70 2024-05-27T07:09:53.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.141+0000 7f97c2000700 10 client.36206 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:09:53.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.142+0000 7f97c2000700 1 -- 172.21.15.16:0/1796479339 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8025040 con 0x562ffbc8de70 2024-05-27T07:09:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.144+0000 7f97c2000700 1 -- 172.21.15.16:0/1796479339 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f97c80051d0 con 0x562ffbc8de70 2024-05-27T07:09:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.144+0000 7f97c2000700 1 -- 172.21.15.16:0/1796479339 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f97c80053d0 con 0x562ffbc8de70 2024-05-27T07:09:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.144+0000 7f97edce4b80 10 client.36206 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:53.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.144+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x562ffbb7c060 con 0x562ffbc8de70 2024-05-27T07:09:53.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.145+0000 7f97c2000700 1 -- 172.21.15.16:0/1796479339 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f97c800b7c0 con 0x562ffbc8de70 2024-05-27T07:09:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 10 client.36206 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:53.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 10 client.36206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 20 client.36206 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbca7880 0x562ffbca9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 4 client.36206 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:09:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.146+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffbbbdd90 con 0x562ffbca7880 2024-05-27T07:09:53.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.147+0000 7f97d7e00700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbca7880 0x562ffbca9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.148+0000 7f97c2000700 10 client.36206 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.148+0000 7f97d7e00700 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbca7880 0x562ffbca9ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4717141adbf6aead server_cookie=ce0eddeb2bdecac8 in_seq=0 out_seq=0 2024-05-27T07:09:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.151+0000 7f97c2000700 1 -- 172.21.15.16:0/1796479339 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ffbbbdd90 con 0x562ffbca7880 2024-05-27T07:09:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.151+0000 7f97c2000700 10 client.36206 handle_command_reply: tid=0 2024-05-27T07:09:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.151+0000 7f97edce4b80 1 client.36206 shutdown 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.151+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 msgr2=0x562ffbc98100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.151+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009810 tx=0x7f97c800f880 comp rx=0 tx=0).stop 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 shutdown_connections 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbca7880 0x562ffbca9ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbc98100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbb0ee40 0x562ffbc97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 --2- 172.21.15.16:0/1796479339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbbfa9f0 0x562ffbc97bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97c2000700 0 client.36206 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.152+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 >> 172.21.15.16:0/1796479339 conn(0x562ffbba8390 msgr2=0x562ffbc8d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.153+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 shutdown_connections 2024-05-27T07:09:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.153+0000 7f97edce4b80 1 -- 172.21.15.16:0/1796479339 wait complete. 2024-05-27T07:09:53.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.154+0000 7f97edce4b80 20 client.36206 trim_cache size 0 max 16384 2024-05-27T07:09:53.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.205+0000 7f97edce4b80 1 Processor -- start 2024-05-27T07:09:53.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.205+0000 7f97edce4b80 1 -- start start 2024-05-27T07:09:53.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbb0ee40 0x562ffbc92e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 0x562ffbca3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81bf290 con 0x562ffbc8de70 2024-05-27T07:09:53.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81c0dd0 con 0x562ffbb0ee40 2024-05-27T07:09:53.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.206+0000 7f97edce4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81bf960 con 0x562ffbbfa9f0 2024-05-27T07:09:53.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.207+0000 7f97dd200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 0x562ffbca3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.207+0000 7f97d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.207+0000 7f97d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36816/0 (socket says 172.21.15.16:36816) 2024-05-27T07:09:53.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.207+0000 7f97d7400700 1 -- 172.21.15.16:0/2637503748 learned_addr learned my addr 172.21.15.16:0/2637503748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.207+0000 7f97d7e00700 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbb0ee40 0x562ffbc92e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.208+0000 7f97d7400700 1 -- 172.21.15.16:0/2637503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbb0ee40 msgr2=0x562ffbc92e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.208+0000 7f97d7400700 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbb0ee40 0x562ffbc92e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.208+0000 7f97d7400700 1 -- 172.21.15.16:0/2637503748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 msgr2=0x562ffbca3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.208+0000 7f97d7400700 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 0x562ffbca3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.208+0000 7f97d7400700 1 -- 172.21.15.16:0/2637503748 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffbc8e800 con 0x562ffbbfa9f0 2024-05-27T07:09:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.211+0000 7f97dd200700 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 0x562ffbca3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:09:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.212+0000 7f97d7400700 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97d00a8ec0 tx=0x7f97d00a91c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a684e74924c7ee19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.212+0000 7f97ccc00700 1 -- 172.21.15.16:0/2637503748 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00c3070 con 0x562ffbbfa9f0 2024-05-27T07:09:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.212+0000 7f97ccc00700 1 -- 172.21.15.16:0/2637503748 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d00ad570 con 0x562ffbbfa9f0 2024-05-27T07:09:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.212+0000 7f97ccc00700 1 -- 172.21.15.16:0/2637503748 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00a9640 con 0x562ffbbfa9f0 2024-05-27T07:09:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.214+0000 7f97edce4b80 1 -- 172.21.15.16:0/2637503748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 msgr2=0x562ffbc98070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.214+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97d00a8ec0 tx=0x7f97d00a91c0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 -- 172.21.15.16:0/2637503748 shutdown_connections 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbc8de70 0x562ffbca3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbb0ee40 0x562ffbc92e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2637503748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbc98070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 -- 172.21.15.16:0/2637503748 >> 172.21.15.16:0/2637503748 conn(0x562ffbba8390 msgr2=0x562ffbc8d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 -- 172.21.15.16:0/2637503748 shutdown_connections 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.215+0000 7f97edce4b80 1 -- 172.21.15.16:0/2637503748 wait complete. 2024-05-27T07:09:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.216+0000 7f97edce4b80 1 -- start start 2024-05-27T07:09:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.217+0000 7f97edce4b80 1 Processor -- start 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbcad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbca8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbca8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81c0700 con 0x562ffbb0ee40 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81c0030 con 0x562ffbc8de70 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81a41f0 con 0x562ffbbfa9f0 2024-05-27T07:09:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.218+0000 7f97edce4b80 10 client.0 _ll_register_callbacks cb 0x562ffbb55c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbca8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbca8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbca8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36828/0 (socket says 172.21.15.16:36828) 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97d7400700 1 -- 172.21.15.16:0/2166618739 learned_addr learned my addr 172.21.15.16:0/2166618739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97d7e00700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbcad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 -- 172.21.15.16:0/2166618739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 msgr2=0x562ffbca8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbca8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 -- 172.21.15.16:0/2166618739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 msgr2=0x562ffbcad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:09:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbcad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.219+0000 7f97dd200700 1 -- 172.21.15.16:0/2166618739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffbca9220 con 0x562ffbc8de70 2024-05-27T07:09:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.220+0000 7f97dd200700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbca8bb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009590 tx=0x7f97c80057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cea71797401bff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:09:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.220+0000 7f97c2000700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:09:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.221+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c803b060 con 0x562ffbc8de70 2024-05-27T07:09:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.222+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f97c800bd10 con 0x562ffbc8de70 2024-05-27T07:09:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.222+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d00ad570 con 0x562ffbc8de70 2024-05-27T07:09:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.223+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f97c8025070 con 0x562ffbc8de70 2024-05-27T07:09:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.223+0000 7f97edce4b80 10 client.36212 fetch_fsmap learned FSMap version 31 2024-05-27T07:09:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.223+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562ffbd11ef0 con 0x562ffbc8de70 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.223+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d00ad570 con 0x562ffbc8de70 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f97c8021480 con 0x562ffbc8de70 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 10 client.36212 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 10 client.36212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 20 client.36212 populate_metadata read hostname 'smithi016' 2024-05-27T07:09:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbd124d0 0x562ffbd148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:09:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 4 client.36212 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:09:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.225+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ffbbb7570 con 0x562ffbd124d0 2024-05-27T07:09:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.226+0000 7f97d7e00700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbd124d0 0x562ffbd148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:09:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.227+0000 7f97d7e00700 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbd124d0 0x562ffbd148f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d763d63ad97f8b7b server_cookie=eceb32dc57074e65 in_seq=0 out_seq=0 2024-05-27T07:09:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.230+0000 7f97c2000700 10 client.36212 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.230+0000 7f97c2000700 1 -- 172.21.15.16:0/2166618739 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ffbbb7570 con 0x562ffbd124d0 2024-05-27T07:09:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.230+0000 7f97c2000700 10 client.36212 handle_command_reply: tid=0 2024-05-27T07:09:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.230+0000 7f97edce4b80 1 client.36212 shutdown 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 msgr2=0x562ffbca8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbca8bb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009590 tx=0x7f97c80057a0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 shutdown_connections 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x562ffbd124d0 0x562ffbd148f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ffbc8de70 0x562ffbca8bb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ffbb0ee40 0x562ffbcad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 --2- 172.21.15.16:0/2166618739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ffbbfa9f0 0x562ffbca8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 >> 172.21.15.16:0/2166618739 conn(0x562ffbba8390 msgr2=0x562ffbc8d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.233+0000 7f97c2000700 0 client.36212 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:09:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.234+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 shutdown_connections 2024-05-27T07:09:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.234+0000 7f97edce4b80 1 -- 172.21.15.16:0/2166618739 wait complete. 2024-05-27T07:09:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.234+0000 7f97edce4b80 20 client.36212 trim_cache size 0 max 16384 2024-05-27T07:09:53.227 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:09:53.227 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:09:53.227 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:09:53.227 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.235+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 msgr2=0x7f97bc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.235+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 0x7f97bc0b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97c4005950 tx=0x7f97c40058e0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.235+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 msgr2=0x7f97d8078200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.235+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007ee0 tx=0x7f97d000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 shutdown_connections 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f97bc0b2af0 0x7f97bc0b5000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97d810c5a0 0x7f97d8078740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d8106560 0x7f97d8077cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 --2- 172.21.15.16:0/866360243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97d8106eb0 0x7f97d8078200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:09:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.236+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 >> 172.21.15.16:0/866360243 conn(0x7f97d8102040 msgr2=0x7f97d8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:09:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.237+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 shutdown_connections 2024-05-27T07:09:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:09:53.237+0000 7f97df200700 1 -- 172.21.15.16:0/866360243 wait complete. 2024-05-27T07:09:54.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:53 smithi016 ceph-mon[31415]: pgmap v277: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 31 MiB/s wr, 609 op/s 2024-05-27T07:09:54.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:53 smithi164 ceph-mon[35327]: pgmap v277: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 31 MiB/s wr, 609 op/s 2024-05-27T07:09:54.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:53 smithi069 ceph-mon[35276]: pgmap v277: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 31 MiB/s wr, 609 op/s 2024-05-27T07:09:54.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:09:54.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:09:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:09:56.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:55 smithi164 ceph-mon[35327]: pgmap v278: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 22 MiB/s wr, 433 op/s 2024-05-27T07:09:56.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:55 smithi016 ceph-mon[31415]: pgmap v278: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 22 MiB/s wr, 433 op/s 2024-05-27T07:09:56.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:55 smithi069 ceph-mon[35276]: pgmap v278: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 22 MiB/s wr, 433 op/s 2024-05-27T07:09:57.233 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:56 smithi164 ceph-mon[35327]: pgmap v279: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 28 MiB/s wr, 558 op/s 2024-05-27T07:09:57.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:56 smithi016 ceph-mon[31415]: pgmap v279: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 28 MiB/s wr, 558 op/s 2024-05-27T07:09:57.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:56 smithi069 ceph-mon[35276]: pgmap v279: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 28 MiB/s wr, 558 op/s 2024-05-27T07:09:59.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:09:59 smithi164 ceph-mon[35327]: pgmap v280: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 28 MiB/s wr, 538 op/s 2024-05-27T07:09:59.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:09:59 smithi016 ceph-mon[31415]: pgmap v280: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 28 MiB/s wr, 538 op/s 2024-05-27T07:09:59.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:09:59 smithi069 ceph-mon[35276]: pgmap v280: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 28 MiB/s wr, 538 op/s 2024-05-27T07:10:00.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:00 smithi164 ceph-mon[35327]: overall HEALTH_OK 2024-05-27T07:10:00.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:00 smithi016 ceph-mon[31415]: overall HEALTH_OK 2024-05-27T07:10:00.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:00 smithi069 ceph-mon[35276]: overall HEALTH_OK 2024-05-27T07:10:01.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:01 smithi164 ceph-mon[35327]: pgmap v281: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-05-27T07:10:01.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:01 smithi016 ceph-mon[31415]: pgmap v281: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-05-27T07:10:01.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:01 smithi069 ceph-mon[35276]: pgmap v281: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-05-27T07:10:03.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:03 smithi016 ceph-mon[31415]: pgmap v282: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 632 op/s 2024-05-27T07:10:03.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:03 smithi164 ceph-mon[35327]: pgmap v282: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 632 op/s 2024-05-27T07:10:03.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:03 smithi069 ceph-mon[35276]: pgmap v282: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 632 op/s 2024-05-27T07:10:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.464+0000 7f6d9ac00700 1 -- 172.21.15.16:0/958362664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0948a0 msgr2=0x7f6d8c094cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.464+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/958362664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0948a0 0x7f6d8c094cb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098f0 tx=0x7f6d88009c00 comp rx=0 tx=0).stop 2024-05-27T07:10:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 -- 172.21.15.16:0/958362664 shutdown_connections 2024-05-27T07:10:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/958362664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c0a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/958362664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0948a0 0x7f6d8c094cb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/958362664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c0951f0 0x7f6d8c09e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 -- 172.21.15.16:0/958362664 >> 172.21.15.16:0/958362664 conn(0x7f6d8c00a380 msgr2=0x7f6d8c00a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 -- 172.21.15.16:0/958362664 shutdown_connections 2024-05-27T07:10:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.465+0000 7f6d9ac00700 1 -- 172.21.15.16:0/958362664 wait complete. 2024-05-27T07:10:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.467+0000 7f6d9ac00700 1 Processor -- start 2024-05-27T07:10:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.467+0000 7f6d9ac00700 1 -- start start 2024-05-27T07:10:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.468+0000 7f6d9ac00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d8c0948a0 0x7f6d8c1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.469+0000 7f6d9ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0951f0 0x7f6d8c140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.469+0000 7f6d9ac00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.469+0000 7f6d9ac00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c0a8880 con 0x7f6d8c0948a0 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.469+0000 7f6d9ac00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c0a8700 con 0x7f6d8c09ed00 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.469+0000 7f6d9ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c0a8a00 con 0x7f6d8c0951f0 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37156/0 (socket says 172.21.15.16:37156) 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 -- 172.21.15.16:0/3626114588 learned_addr learned my addr 172.21.15.16:0/3626114588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d98e00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0951f0 0x7f6d8c140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 -- 172.21.15.16:0/3626114588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0951f0 msgr2=0x7f6d8c140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0951f0 0x7f6d8c140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 -- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d8c0948a0 msgr2=0x7f6d8c1402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d8c0948a0 0x7f6d8c1402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.470+0000 7f6d9a200700 1 -- 172.21.15.16:0/3626114588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c145170 con 0x7f6d8c09ed00 2024-05-27T07:10:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.472+0000 7f6d9a200700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000eb0 tx=0x7f6d9407b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:04.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.473+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9406ab30 con 0x7f6d8c09ed00 2024-05-27T07:10:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.473+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d9407bd70 con 0x7f6d8c09ed00 2024-05-27T07:10:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.474+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9406c5e0 con 0x7f6d8c09ed00 2024-05-27T07:10:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.474+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c145460 con 0x7f6d8c09ed00 2024-05-27T07:10:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.474+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c133d80 con 0x7f6d8c09ed00 2024-05-27T07:10:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.476+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c095b20 con 0x7f6d8c09ed00 2024-05-27T07:10:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.489+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6d9406acd0 con 0x7f6d8c09ed00 2024-05-27T07:10:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.492+0000 7f6d92000700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 0x7f6d7c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.492+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f6d94147cf0 con 0x7f6d8c09ed00 2024-05-27T07:10:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.493+0000 7f6d99800700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 0x7f6d7c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.493+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d94148210 con 0x7f6d8c09ed00 2024-05-27T07:10:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:04.495+0000 7f6d99800700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 0x7f6d7c0b4fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009680 tx=0x7f6d88009f50 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:05.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.075+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d8c141600 con 0x7f6d8c09ed00 2024-05-27T07:10:05.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.076+0000 7f6d92000700 1 -- 172.21.15.16:0/3626114588 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f6d9410f2e0 con 0x7f6d8c09ed00 2024-05-27T07:10:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:10:05.070 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:10:05.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 msgr2=0x7f6d7c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:05.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 0x7f6d7c0b4fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009680 tx=0x7f6d88009f50 comp rx=0 tx=0).stop 2024-05-27T07:10:05.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 msgr2=0x7f6d8c144ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:05.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000eb0 tx=0x7f6d9407b260 comp rx=0 tx=0).stop 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 shutdown_connections 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f6d7c0b2ab0 0x7f6d7c0b4fc0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009680 tx=0x7f6d88009f50 comp rx=0 tx=0).stop 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8c09ed00 0x7f6d8c144ad0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d8c0948a0 0x7f6d8c1402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 --2- 172.21.15.16:0/3626114588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d8c0951f0 0x7f6d8c140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.084+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 >> 172.21.15.16:0/3626114588 conn(0x7f6d8c00a380 msgr2=0x7f6d8c096260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.085+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 shutdown_connections 2024-05-27T07:10:05.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:05.086+0000 7f6d9ac00700 1 -- 172.21.15.16:0/3626114588 wait complete. 2024-05-27T07:10:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:10:05.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:05 smithi016 ceph-mon[31415]: pgmap v283: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 489 op/s 2024-05-27T07:10:05.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:05 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3626114588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:10:05.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:05 smithi069 ceph-mon[35276]: pgmap v283: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 489 op/s 2024-05-27T07:10:05.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:05 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3626114588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:10:05.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:05 smithi164 ceph-mon[35327]: pgmap v283: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 489 op/s 2024-05-27T07:10:05.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:05 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3626114588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:10:07.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:07 smithi016 ceph-mon[31415]: pgmap v284: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 533 op/s 2024-05-27T07:10:07.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:07 smithi069 ceph-mon[35276]: pgmap v284: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 533 op/s 2024-05-27T07:10:07.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:07 smithi164 ceph-mon[35327]: pgmap v284: 129 pgs: 129 active+clean; 19 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 533 op/s 2024-05-27T07:10:08.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:10:09.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:09 smithi016 ceph-mon[31415]: pgmap v285: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:09.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:09 smithi069 ceph-mon[35276]: pgmap v285: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:09.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:09 smithi164 ceph-mon[35327]: pgmap v285: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:09.633 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:10:10.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:10 smithi016 ceph-mon[31415]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:10:10.598 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:10 smithi164 ceph-mon[35327]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:10:10.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:10 smithi069 ceph-mon[35276]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:10:11.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:11 smithi016 ceph-mon[31415]: pgmap v287: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 511 op/s 2024-05-27T07:10:11.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:11 smithi164 ceph-mon[35327]: pgmap v287: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 511 op/s 2024-05-27T07:10:11.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:11 smithi069 ceph-mon[35276]: pgmap v287: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 511 op/s 2024-05-27T07:10:13.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:13 smithi164 ceph-mon[35327]: pgmap v288: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 420 op/s 2024-05-27T07:10:13.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:13 smithi016 ceph-mon[31415]: pgmap v288: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 420 op/s 2024-05-27T07:10:13.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:13 smithi069 ceph-mon[35276]: pgmap v288: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 420 op/s 2024-05-27T07:10:15.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:15 smithi164 ceph-mon[35327]: pgmap v289: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-05-27T07:10:15.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:15 smithi016 ceph-mon[31415]: pgmap v289: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-05-27T07:10:15.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:15 smithi069 ceph-mon[35276]: pgmap v289: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-05-27T07:10:17.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:17 smithi164 ceph-mon[35327]: pgmap v290: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-27T07:10:17.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:17 smithi016 ceph-mon[31415]: pgmap v290: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-27T07:10:17.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:17 smithi069 ceph-mon[35276]: pgmap v290: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-27T07:10:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.847+0000 7f0be5000700 1 -- 172.21.15.16:0/1601119918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be00764a0 msgr2=0x7f0be0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 --2- 172.21.15.16:0/1601119918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be00764a0 0x7f0be0114e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0098f0 tx=0x7f0bcc009c00 comp rx=0 tx=0).stop 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 -- 172.21.15.16:0/1601119918 shutdown_connections 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 --2- 172.21.15.16:0/1601119918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be0115360 0x7f0be0117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 --2- 172.21.15.16:0/1601119918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0075b50 0x7f0be0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 --2- 172.21.15.16:0/1601119918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be00764a0 0x7f0be0114e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 -- 172.21.15.16:0/1601119918 >> 172.21.15.16:0/1601119918 conn(0x7f0be0100030 msgr2=0x7f0be0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 -- 172.21.15.16:0/1601119918 shutdown_connections 2024-05-27T07:10:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.848+0000 7f0be5000700 1 -- 172.21.15.16:0/1601119918 wait complete. 2024-05-27T07:10:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.850+0000 7f0be5000700 1 Processor -- start 2024-05-27T07:10:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.850+0000 7f0be5000700 1 -- start start 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be00764a0 0x7f0be01a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 0x7f0be01a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be011a9b0 con 0x7f0be00764a0 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0be011a830 con 0x7f0be0075b50 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0be5000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0be011ab30 con 0x7f0be0115360 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:49560/0 (socket says 172.21.15.16:49560) 2024-05-27T07:10:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0bdde00700 1 -- 172.21.15.16:0/4203102153 learned_addr learned my addr 172.21.15.16:0/4203102153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.851+0000 7f0bde800700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 0x7f0be01a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be00764a0 0x7f0be01a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 -- 172.21.15.16:0/4203102153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 msgr2=0x7f0be01a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 0x7f0be01a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 -- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be00764a0 msgr2=0x7f0be01a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be00764a0 0x7f0be01a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 -- 172.21.15.16:0/4203102153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01a6200 con 0x7f0be0075b50 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bde800700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 0x7f0be01a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.852+0000 7f0bdde00700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e990 tx=0x7f0bd000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.853+0000 7f0bd6a00700 1 -- 172.21.15.16:0/4203102153 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004220 con 0x7f0be0075b50 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.853+0000 7f0bd6a00700 1 -- 172.21.15.16:0/4203102153 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd00043c0 con 0x7f0be0075b50 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.853+0000 7f0bd6a00700 1 -- 172.21.15.16:0/4203102153 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0005650 con 0x7f0be0075b50 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.853+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01a64f0 con 0x7f0be0075b50 2024-05-27T07:10:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.853+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a69e0 con 0x7f0be0075b50 2024-05-27T07:10:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.866+0000 7f0bd6a00700 1 -- 172.21.15.16:0/4203102153 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0bd000c760 con 0x7f0be0075b50 2024-05-27T07:10:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.868+0000 7f0bd6a00700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 0x7f0bc40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.869+0000 7f0bd6a00700 1 -- 172.21.15.16:0/4203102153 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9613+0+0 (secure 0 0 0) 0x7f0bd00e3610 con 0x7f0be0075b50 2024-05-27T07:10:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.869+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 0x7f0bc40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.881+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 0x7f0bc40b4fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009680 tx=0x7f0bcc005e40 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.884+0000 7f0bf3b38b80 1 Processor -- start 2024-05-27T07:10:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.884+0000 7f0bf3b38b80 1 -- start start 2024-05-27T07:10:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b8d30 0x55fae08b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 0x55fae08b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fae07e0e40 con 0x55fae0733e40 2024-05-27T07:10:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fae07d09e0 con 0x55fae08b8d30 2024-05-27T07:10:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fae07d4a20 con 0x55fae08b2db0 2024-05-27T07:10:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56968/0 (socket says 172.21.15.16:56968) 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.885+0000 7f0bdde00700 1 -- 172.21.15.16:0/1041582909 learned_addr learned my addr 172.21.15.16:0/1041582909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bde800700 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 0x55fae08b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bdde00700 1 -- 172.21.15.16:0/1041582909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 msgr2=0x55fae08b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bdde00700 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 0x55fae08b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bdde00700 1 -- 172.21.15.16:0/1041582909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b8d30 msgr2=0x55fae08b2870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bdde00700 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b8d30 0x55fae08b2870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.886+0000 7f0bdde00700 1 -- 172.21.15.16:0/1041582909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae0799420 con 0x55fae0733e40 2024-05-27T07:10:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.887+0000 7f0bde800700 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 0x55fae08b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:10:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.887+0000 7f0bdde00700 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e960 tx=0x7f0bd00aa110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4ff5ec8df672ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.888+0000 7f0bcb400700 1 -- 172.21.15.16:0/1041582909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd001f040 con 0x55fae0733e40 2024-05-27T07:10:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.888+0000 7f0bcb400700 1 -- 172.21.15.16:0/1041582909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd00b3310 con 0x55fae0733e40 2024-05-27T07:10:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.888+0000 7f0bcb400700 1 -- 172.21.15.16:0/1041582909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00aa710 con 0x55fae0733e40 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.889+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/1041582909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 msgr2=0x55fae08b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.889+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e960 tx=0x7f0bd00aa110 comp rx=0 tx=0).stop 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.891+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/1041582909 shutdown_connections 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.891+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b2db0 0x55fae08b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.891+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08b87f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.891+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/1041582909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b8d30 0x55fae08b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.891+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/1041582909 >> 172.21.15.16:0/1041582909 conn(0x55fae0793fc0 msgr2=0x55fae08ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.892+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/1041582909 shutdown_connections 2024-05-27T07:10:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.892+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/1041582909 wait complete. 2024-05-27T07:10:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.893+0000 7f0bf3b38b80 1 -- start start 2024-05-27T07:10:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.893+0000 7f0bf3b38b80 1 Processor -- start 2024-05-27T07:10:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.894+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.895+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fae06d6fa0 con 0x55fae0733e40 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fae0771760 con 0x55fae08b2db0 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fae08bf800 con 0x55fae08b8d30 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bf3b38b80 10 client.0 _ll_register_callbacks cb 0x55fae077ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bde800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bde800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58634/0 (socket says 172.21.15.16:58634) 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.896+0000 7f0bde800700 1 -- 172.21.15.16:0/30393430 learned_addr learned my addr 172.21.15.16:0/30393430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bdd400700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bde800700 1 -- 172.21.15.16:0/30393430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 msgr2=0x55fae08bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bde800700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bdde00700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bde800700 1 -- 172.21.15.16:0/30393430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 msgr2=0x55fae08bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bde800700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bde800700 1 -- 172.21.15.16:0/30393430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae07208d0 con 0x55fae08b8d30 2024-05-27T07:10:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bdd400700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.897+0000 7f0bdde00700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.898+0000 7f0bde800700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ff70 tx=0x7f0bd80115e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727fee929011ab8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.898+0000 7f0bbfe00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.898+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8019070 con 0x55fae08b8d30 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.898+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9613+0+0 (secure 0 0 0) 0x7f0bd8010040 con 0x55fae08b8d30 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.898+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd00b3310 con 0x55fae08b8d30 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.899+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0bd8024050 con 0x55fae08b8d30 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.899+0000 7f0bf3b38b80 10 client.26155 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.899+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fae07a1060 con 0x55fae08b8d30 2024-05-27T07:10:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.900+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd00b3310 con 0x55fae08b8d30 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.901+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0bd80144f0 con 0x55fae08b8d30 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.901+0000 7f0bf3b38b80 10 client.26155 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.901+0000 7f0bf3b38b80 10 client.26155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.901+0000 7f0bf3b38b80 20 client.26155 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.902+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae08cc7a0 0x55fae08cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.902+0000 7f0bf3b38b80 4 client.26155 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:10:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.902+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae07e2d90 con 0x55fae08cc7a0 2024-05-27T07:10:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.903+0000 7f0bdde00700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae08cc7a0 0x55fae08cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.906+0000 7f0bdde00700 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae08cc7a0 0x55fae08cec30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93ad8daaffabd774 server_cookie=c99c99b969f1bb6a in_seq=0 out_seq=0 2024-05-27T07:10:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.908+0000 7f0bbfe00700 10 client.26155 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:10:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.908+0000 7f0bbfe00700 1 -- 172.21.15.16:0/30393430 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fae07e2d90 con 0x55fae08cc7a0 2024-05-27T07:10:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.908+0000 7f0bbfe00700 10 client.26155 handle_command_reply: tid=0 2024-05-27T07:10:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.908+0000 7f0bf3b38b80 1 client.26155 shutdown 2024-05-27T07:10:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 msgr2=0x55fae08bd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ff70 tx=0x7f0bd80115e0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 shutdown_connections 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae08cc7a0 0x55fae08cec30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08b8d30 0x55fae08bd000 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae0733e40 0x55fae08bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/30393430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 >> 172.21.15.16:0/30393430 conn(0x55fae0793fc0 msgr2=0x55fae08ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.909+0000 7f0bbfe00700 0 client.26155 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:10:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.910+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 shutdown_connections 2024-05-27T07:10:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.910+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/30393430 wait complete. 2024-05-27T07:10:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.911+0000 7f0bf3b38b80 20 client.26155 trim_cache size 0 max 16384 2024-05-27T07:10:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.946+0000 7f0bf3b38b80 1 Processor -- start 2024-05-27T07:10:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.947+0000 7f0bf3b38b80 1 -- start start 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc000ef60 con 0x55fae08b8d30 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0010aa0 con 0x55fae08b2db0 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc000f630 con 0x55fae0733e40 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.948+0000 7f0bdd400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.949+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.949+0000 7f0bdde00700 1 -- 172.21.15.16:0/4153513107 learned_addr learned my addr 172.21.15.16:0/4153513107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.949+0000 7f0bdd400700 1 -- 172.21.15.16:0/4153513107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 msgr2=0x55fae08b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bdd400700 1 -- 172.21.15.16:0/4153513107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 msgr2=0x55fae08b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bde800700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bdd400700 1 -- 172.21.15.16:0/4153513107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae08bab10 con 0x55fae08b2db0 2024-05-27T07:10:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bdde00700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:10:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.950+0000 7f0bde800700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.951+0000 7f0bdd400700 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08c88d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0be01036c0 tx=0x7f0bcc01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9be750e07013ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.951+0000 7f0bcaa00700 1 -- 172.21.15.16:0/4153513107 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00f9a0 con 0x55fae08b2db0 2024-05-27T07:10:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.951+0000 7f0bcaa00700 1 -- 172.21.15.16:0/4153513107 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bcc0054d0 con 0x55fae08b2db0 2024-05-27T07:10:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.951+0000 7f0bcaa00700 1 -- 172.21.15.16:0/4153513107 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00b610 con 0x55fae08b2db0 2024-05-27T07:10:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/4153513107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 msgr2=0x55fae08c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08c88d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0be01036c0 tx=0x7f0bcc01d000 comp rx=0 tx=0).stop 2024-05-27T07:10:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/4153513107 shutdown_connections 2024-05-27T07:10:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae0733e40 0x55fae08b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/4153513107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae08b2db0 0x55fae08c88d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.953+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/4153513107 >> 172.21.15.16:0/4153513107 conn(0x55fae0793fc0 msgr2=0x55fae08b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.954+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/4153513107 shutdown_connections 2024-05-27T07:10:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.954+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/4153513107 wait complete. 2024-05-27T07:10:19.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:19 smithi016 ceph-mon[31415]: pgmap v291: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 383 op/s 2024-05-27T07:10:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.955+0000 7f0bf3b38b80 1 -- start start 2024-05-27T07:10:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.955+0000 7f0bf3b38b80 1 Processor -- start 2024-05-27T07:10:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.957+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae0733e40 0x55fae08c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.957+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bf3b38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08d33e0 0x55fae08d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.958+0000 7f0bdde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae0733e40 0x55fae08c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00103d0 con 0x55fae08b8d30 2024-05-27T07:10:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bf3b38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc000fd00 con 0x55fae0733e40 2024-05-27T07:10:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bdd400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bdd400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56982/0 (socket says 172.21.15.16:56982) 2024-05-27T07:10:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.960+0000 7f0bdd400700 1 -- 172.21.15.16:0/834865740 learned_addr learned my addr 172.21.15.16:0/834865740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.961+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc000cf20 con 0x55fae08d33e0 2024-05-27T07:10:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.961+0000 7f0bde800700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08d33e0 0x55fae08d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.963+0000 7f0bf3b38b80 10 client.0 _ll_register_callbacks cb 0x55fae077ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.964+0000 7f0bdd400700 1 -- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08d33e0 msgr2=0x55fae08d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.964+0000 7f0bdd400700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08d33e0 0x55fae08d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.964+0000 7f0bdd400700 1 -- 172.21.15.16:0/834865740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae0733e40 msgr2=0x55fae08c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.964+0000 7f0bdd400700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae0733e40 0x55fae08c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.964+0000 7f0bdd400700 1 -- 172.21.15.16:0/834865740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae08d3d30 con 0x55fae08b8d30 2024-05-27T07:10:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.966+0000 7f0bdd400700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0be01036c0 tx=0x7f0bcc0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6a00f509255c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.966+0000 7f0bbfe00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:10:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.969+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0048e0 con 0x55fae08b8d30 2024-05-27T07:10:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.969+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc0054d0 con 0x55fae08b8d30 2024-05-27T07:10:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.970+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9613+0+0 (secure 0 0 0) 0x7f0bcc004b80 con 0x55fae08b8d30 2024-05-27T07:10:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.970+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc03e050 con 0x55fae08b8d30 2024-05-27T07:10:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.970+0000 7f0bf3b38b80 10 client.16341 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.970+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fae0936cb0 con 0x55fae08b8d30 2024-05-27T07:10:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.971+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bcc0054d0 con 0x55fae08b8d30 2024-05-27T07:10:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.972+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0bcc00baf0 con 0x55fae08b8d30 2024-05-27T07:10:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 10 client.16341 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 20 client.16341 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae0937290 0x55fae09396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 4 client.16341 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.976+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fae07dc570 con 0x55fae0937290 2024-05-27T07:10:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.977+0000 7f0bdde00700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae0937290 0x55fae09396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.979+0000 7f0bdde00700 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae0937290 0x55fae09396b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72aa28dbb81a2175 server_cookie=4c8e91ec15e34758 in_seq=0 out_seq=0 2024-05-27T07:10:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bbfe00700 10 client.16341 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:10:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bbfe00700 1 -- 172.21.15.16:0/834865740 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fae07dc570 con 0x55fae0937290 2024-05-27T07:10:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bbfe00700 10 client.16341 handle_command_reply: tid=0 2024-05-27T07:10:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 client.16341 shutdown 2024-05-27T07:10:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 msgr2=0x55fae08c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0be01036c0 tx=0x7f0bcc0056f0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 shutdown_connections 2024-05-27T07:10:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x55fae0937290 0x55fae09396b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fae08d33e0 0x55fae08d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bbfe00700 0 client.16341 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fae0733e40 0x55fae08c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 --2- 172.21.15.16:0/834865740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae08b8d30 0x55fae08c5000 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.980+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 >> 172.21.15.16:0/834865740 conn(0x55fae0793fc0 msgr2=0x55fae08bc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.983+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 shutdown_connections 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.983+0000 7f0bf3b38b80 1 -- 172.21.15.16:0/834865740 wait complete. 2024-05-27T07:10:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.983+0000 7f0bf3b38b80 20 client.16341 trim_cache size 0 max 16384 2024-05-27T07:10:19.974 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:10:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.984+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 msgr2=0x7f0bc40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.984+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 0x7f0bc40b4fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009680 tx=0x7f0bcc005e40 comp rx=0 tx=0).stop 2024-05-27T07:10:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.984+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 msgr2=0x7f0be01a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.984+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e990 tx=0x7f0bd000eca0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 shutdown_connections 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0bc40b2aa0 0x7f0bc40b4fb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0115360 0x7f0be01a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be0075b50 0x7f0be01a13e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 --2- 172.21.15.16:0/4203102153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be00764a0 0x7f0be01a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 >> 172.21.15.16:0/4203102153 conn(0x7f0be0100030 msgr2=0x7f0be0101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 shutdown_connections 2024-05-27T07:10:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:19.985+0000 7f0be5000700 1 -- 172.21.15.16:0/4203102153 wait complete. 2024-05-27T07:10:20.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:19 smithi164 ceph-mon[35327]: pgmap v291: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 383 op/s 2024-05-27T07:10:20.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:19 smithi069 ceph-mon[35276]: pgmap v291: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 383 op/s 2024-05-27T07:10:21.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:10:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:21 smithi164 ceph-mon[35327]: pgmap v292: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 353 op/s 2024-05-27T07:10:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:21 smithi016 ceph-mon[31415]: pgmap v292: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 353 op/s 2024-05-27T07:10:22.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:21 smithi069 ceph-mon[35276]: pgmap v292: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 353 op/s 2024-05-27T07:10:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:10:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:23 smithi164 ceph-mon[35327]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 468 op/s 2024-05-27T07:10:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:23 smithi164 ceph-mon[35327]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:10:24.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:23 smithi016 ceph-mon[31415]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 468 op/s 2024-05-27T07:10:24.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:23 smithi016 ceph-mon[31415]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:10:24.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:23 smithi069 ceph-mon[35276]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 468 op/s 2024-05-27T07:10:24.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:23 smithi069 ceph-mon[35276]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:10:26.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:25 smithi164 ceph-mon[35327]: pgmap v295: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 22 MiB/s wr, 450 op/s 2024-05-27T07:10:26.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:25 smithi016 ceph-mon[31415]: pgmap v295: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 22 MiB/s wr, 450 op/s 2024-05-27T07:10:26.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:25 smithi069 ceph-mon[35276]: pgmap v295: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 22 MiB/s wr, 450 op/s 2024-05-27T07:10:28.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:27 smithi164 ceph-mon[35327]: pgmap v296: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 24 MiB/s wr, 486 op/s 2024-05-27T07:10:28.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:27 smithi016 ceph-mon[31415]: pgmap v296: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 24 MiB/s wr, 486 op/s 2024-05-27T07:10:28.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:27 smithi069 ceph-mon[35276]: pgmap v296: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 24 MiB/s wr, 486 op/s 2024-05-27T07:10:29.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:29 smithi016 ceph-mon[31415]: pgmap v297: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 473 op/s 2024-05-27T07:10:29.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:29 smithi016 ceph-mon[31415]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:10:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:29 smithi164 ceph-mon[35327]: pgmap v297: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 473 op/s 2024-05-27T07:10:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:29 smithi164 ceph-mon[35327]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:10:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:29 smithi069 ceph-mon[35276]: pgmap v297: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 473 op/s 2024-05-27T07:10:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:29 smithi069 ceph-mon[35276]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:10:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 -- 172.21.15.16:0/1606827375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd010a920 msgr2=0x7f8dd010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 --2- 172.21.15.16:0/1606827375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd010a920 0x7f8dd010ad30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001c0d0 tx=0x7f8dc001c3e0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 -- 172.21.15.16:0/1606827375 shutdown_connections 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 --2- 172.21.15.16:0/1606827375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd011d920 0x7f8dd011fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 --2- 172.21.15.16:0/1606827375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd010a920 0x7f8dd010ad30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 --2- 172.21.15.16:0/1606827375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd011d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.958+0000 7f8dd6800700 1 -- 172.21.15.16:0/1606827375 >> 172.21.15.16:0/1606827375 conn(0x7f8dd006f4c0 msgr2=0x7f8dd006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.959+0000 7f8dd6800700 1 -- 172.21.15.16:0/1606827375 shutdown_connections 2024-05-27T07:10:30.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.960+0000 7f8dd6800700 1 -- 172.21.15.16:0/1606827375 wait complete. 2024-05-27T07:10:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.963+0000 7f8dd6800700 1 Processor -- start 2024-05-27T07:10:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.964+0000 7f8dd6800700 1 -- start start 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd010a920 0x7f8dd011d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd0119260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 0x7f8dd01197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0115300 con 0x7f8dd010a920 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0115180 con 0x7f8dd011d920 2024-05-27T07:10:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.965+0000 7f8dd6800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0115480 con 0x7f8dd010b270 2024-05-27T07:10:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.966+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 0x7f8dd01197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.966+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 0x7f8dd01197a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:59136/0 (socket says 172.21.15.16:59136) 2024-05-27T07:10:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.966+0000 7f8dcfe00700 1 -- 172.21.15.16:0/283164000 learned_addr learned my addr 172.21.15.16:0/283164000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.966+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd0119260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dcf400700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd010a920 0x7f8dd011d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dc6a00700 1 -- 172.21.15.16:0/283164000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 msgr2=0x7f8dd01197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 0x7f8dd01197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dc6a00700 1 -- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd010a920 msgr2=0x7f8dd011d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd010a920 0x7f8dd011d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.967+0000 7f8dc6a00700 1 -- 172.21.15.16:0/283164000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd0119e70 con 0x7f8dd010b270 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.968+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd0119260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.968+0000 7f8dccc00700 1 -- 172.21.15.16:0/283164000 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8015020 con 0x7f8dd010b270 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.968+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd01ad8b0 con 0x7f8dd010b270 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.968+0000 7f8dccc00700 1 -- 172.21.15.16:0/283164000 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db8011cc0 con 0x7f8dd010b270 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.968+0000 7f8dccc00700 1 -- 172.21.15.16:0/283164000 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8010b80 con 0x7f8dd010b270 2024-05-27T07:10:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.969+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd01add60 con 0x7f8dd010b270 2024-05-27T07:10:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.976+0000 7f8dccc00700 1 -- 172.21.15.16:0/283164000 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8db8018020 con 0x7f8dd010b270 2024-05-27T07:10:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.982+0000 7f8dccc00700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 0x7f8db40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.982+0000 7f8dccc00700 1 -- 172.21.15.16:0/283164000 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f8db80e2ec0 con 0x7f8dd010b270 2024-05-27T07:10:30.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.986+0000 7f8dcf400700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 0x7f8db40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.987+0000 7f8dcf400700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 0x7f8db40b4f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001ae70 tx=0x7f8dc0018f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:30.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.991+0000 7f8de5349b80 1 Processor -- start 2024-05-27T07:10:30.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.991+0000 7f8de5349b80 1 -- start start 2024-05-27T07:10:30.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c4996330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624c48c4e40 con 0x5624c49039f0 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624c48b49e0 con 0x5624c4996e70 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.993+0000 7f8de5349b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5624c48b8a20 con 0x5624c4817e40 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dc6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dc6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42612/0 (socket says 172.21.15.16:42612) 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dcf400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c4996330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:59146/0 (socket says 172.21.15.16:59146) 2024-05-27T07:10:30.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.994+0000 7f8dcfe00700 1 -- 172.21.15.16:0/2023644185 learned_addr learned my addr 172.21.15.16:0/2023644185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.997+0000 7f8dc6a00700 1 -- 172.21.15.16:0/2023644185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 msgr2=0x5624c49a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.997+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.997+0000 7f8dc6a00700 1 -- 172.21.15.16:0/2023644185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 msgr2=0x5624c4996330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.997+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c4996330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.997+0000 7f8dc6a00700 1 -- 172.21.15.16:0/2023644185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c487d420 con 0x5624c4817e40 2024-05-27T07:10:30.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:30.998+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:10:30.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.000+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8db80e02e0 tx=0x7f8db80a9980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac778279029cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:30.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.002+0000 7f8dc4c00700 1 -- 172.21.15.16:0/2023644185 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80ae220 con 0x5624c4817e40 2024-05-27T07:10:30.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.005+0000 7f8dc4c00700 1 -- 172.21.15.16:0/2023644185 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db80b2cd0 con 0x5624c4817e40 2024-05-27T07:10:30.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.006+0000 7f8dc4c00700 1 -- 172.21.15.16:0/2023644185 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80b2aa0 con 0x5624c4817e40 2024-05-27T07:10:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.009+0000 7f8de5349b80 1 -- 172.21.15.16:0/2023644185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 msgr2=0x5624c4996870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.009+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8db80e02e0 tx=0x7f8db80a9980 comp rx=0 tx=0).stop 2024-05-27T07:10:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.009+0000 7f8de5349b80 1 -- 172.21.15.16:0/2023644185 shutdown_connections 2024-05-27T07:10:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.009+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.010+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c4996330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.010+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2023644185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c4996870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.010+0000 7f8de5349b80 1 -- 172.21.15.16:0/2023644185 >> 172.21.15.16:0/2023644185 conn(0x5624c48b1390 msgr2=0x5624c49924c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.010+0000 7f8de5349b80 1 -- 172.21.15.16:0/2023644185 shutdown_connections 2024-05-27T07:10:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.010+0000 7f8de5349b80 1 -- 172.21.15.16:0/2023644185 wait complete. 2024-05-27T07:10:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.012+0000 7f8de5349b80 1 -- start start 2024-05-27T07:10:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.012+0000 7f8de5349b80 1 Processor -- start 2024-05-27T07:10:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.016+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c49a0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.016+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c49a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.016+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.016+0000 7f8de5349b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624c47bafa0 con 0x5624c49039f0 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8de5349b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624c4855760 con 0x5624c4996e70 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8dc6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c49a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:59162/0 (socket says 172.21.15.16:59162) 2024-05-27T07:10:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8dcfe00700 1 -- 172.21.15.16:0/3943973798 learned_addr learned my addr 172.21.15.16:0/3943973798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5624c49a3bd0 con 0x5624c4817e40 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8de5349b80 10 client.0 _ll_register_callbacks cb 0x5624c485ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.017+0000 7f8dcf400700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c49a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8dcfe00700 1 -- 172.21.15.16:0/3943973798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 msgr2=0x5624c49a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c49a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8dcfe00700 1 -- 172.21.15.16:0/3943973798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 msgr2=0x5624c49a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c49a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8dcfe00700 1 -- 172.21.15.16:0/3943973798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c48048d0 con 0x5624c4996e70 2024-05-27T07:10:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.018+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db80b2cd0 con 0x5624c4996e70 2024-05-27T07:10:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.019+0000 7f8dbd600700 10 client.36242 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:10:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.019+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8011360 tx=0x7f8dc8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf13fef1b777de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.020+0000 7f8dbd600700 1 -- 172.21.15.16:0/3943973798 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8023050 con 0x5624c4996e70 2024-05-27T07:10:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.020+0000 7f8dbd600700 1 -- 172.21.15.16:0/3943973798 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f8dc8015870 con 0x5624c4996e70 2024-05-27T07:10:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.020+0000 7f8dbd600700 1 -- 172.21.15.16:0/3943973798 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8dc8019070 con 0x5624c4996e70 2024-05-27T07:10:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 10 client.36242 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5624c4885060 con 0x5624c4996e70 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8dbd600700 1 -- 172.21.15.16:0/3943973798 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8dc8015ad0 con 0x5624c4996e70 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 10 client.36242 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 10 client.36242 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 20 client.36242 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.022+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c49b09f0 0x5624c49b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.023+0000 7f8de5349b80 4 client.36242 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:10:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.023+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c48c6d90 con 0x5624c49b09f0 2024-05-27T07:10:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.023+0000 7f8dcf400700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c49b09f0 0x5624c49b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.024+0000 7f8dcf400700 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c49b09f0 0x5624c49b2e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d2200bf5238a294 server_cookie=bafd596631fe1820 in_seq=0 out_seq=0 2024-05-27T07:10:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.024+0000 7f8dbd600700 10 client.36242 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:10:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.025+0000 7f8dbd600700 1 -- 172.21.15.16:0/3943973798 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624c48c6d90 con 0x5624c49b09f0 2024-05-27T07:10:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.025+0000 7f8dbd600700 10 client.36242 handle_command_reply: tid=0 2024-05-27T07:10:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.025+0000 7f8de5349b80 1 client.36242 shutdown 2024-05-27T07:10:31.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.026+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 msgr2=0x5624c49a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.026+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8011360 tx=0x7f8dc8011720 comp rx=0 tx=0).stop 2024-05-27T07:10:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.026+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 shutdown_connections 2024-05-27T07:10:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.026+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c49b09f0 0x5624c49b2e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4996e70 0x5624c49a10b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4817e40 0x5624c49a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 --2- 172.21.15.16:0/3943973798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c49a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 >> 172.21.15.16:0/3943973798 conn(0x5624c48b1390 msgr2=0x5624c4996310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8dbd600700 0 client.36242 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 shutdown_connections 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 1 -- 172.21.15.16:0/3943973798 wait complete. 2024-05-27T07:10:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.027+0000 7f8de5349b80 20 client.36242 trim_cache size 0 max 16384 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 Processor -- start 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 -- start start 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 0x5624c499be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c49039f0 0x5624c4996720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c49acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b6bb0 con 0x5624c4817e40 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b86f0 con 0x5624c49039f0 2024-05-27T07:10:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.060+0000 7f8de5349b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b7280 con 0x5624c4996e70 2024-05-27T07:10:31.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.061+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c49acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.061+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c49acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42634/0 (socket says 172.21.15.16:42634) 2024-05-27T07:10:31.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.062+0000 7f8dcfe00700 1 -- 172.21.15.16:0/2361679967 learned_addr learned my addr 172.21.15.16:0/2361679967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:31.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.062+0000 7f8dcf400700 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 0x5624c499be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dcf400700 1 -- 172.21.15.16:0/2361679967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 msgr2=0x5624c49acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dcf400700 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c49acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dcf400700 1 -- 172.21.15.16:0/2361679967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c49039f0 msgr2=0x5624c4996720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c49039f0 0x5624c4996720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dcf400700 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c49039f0 0x5624c4996720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.063+0000 7f8dcf400700 1 -- 172.21.15.16:0/2361679967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c4997a50 con 0x5624c4817e40 2024-05-27T07:10:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.065+0000 7f8dcf400700 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 0x5624c499be90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5624c4998ff0 tx=0x7f8dc001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23080a8d3de6fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.066+0000 7f8dbfe00700 1 -- 172.21.15.16:0/2361679967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00423b0 con 0x5624c4817e40 2024-05-27T07:10:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.067+0000 7f8dbfe00700 1 -- 172.21.15.16:0/2361679967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc00429e0 con 0x5624c4817e40 2024-05-27T07:10:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.067+0000 7f8dbfe00700 1 -- 172.21.15.16:0/2361679967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0005620 con 0x5624c4817e40 2024-05-27T07:10:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 -- 172.21.15.16:0/2361679967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 msgr2=0x5624c499be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 0x5624c499be90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5624c4998ff0 tx=0x7f8dc001c040 comp rx=0 tx=0).stop 2024-05-27T07:10:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 -- 172.21.15.16:0/2361679967 shutdown_connections 2024-05-27T07:10:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c49acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c4817e40 0x5624c499be90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 --2- 172.21.15.16:0/2361679967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c49039f0 0x5624c4996720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.068+0000 7f8de5349b80 1 -- 172.21.15.16:0/2361679967 >> 172.21.15.16:0/2361679967 conn(0x5624c48b1390 msgr2=0x5624c499ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.070+0000 7f8de5349b80 1 -- 172.21.15.16:0/2361679967 shutdown_connections 2024-05-27T07:10:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.070+0000 7f8de5349b80 1 -- 172.21.15.16:0/2361679967 wait complete. 2024-05-27T07:10:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.070+0000 7f8de5349b80 1 -- start start 2024-05-27T07:10:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.071+0000 7f8de5349b80 1 Processor -- start 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.072+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 0x5624c499a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.072+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c499ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.072+0000 7f8de5349b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8de5349b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b8020 con 0x5624c49039f0 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8de5349b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b7950 con 0x5624c4817e40 2024-05-27T07:10:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8de5349b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b4b70 con 0x5624c4996e70 2024-05-27T07:10:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8de5349b80 10 client.0 _ll_register_callbacks cb 0x5624c485ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8dcfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42644/0 (socket says 172.21.15.16:42644) 2024-05-27T07:10:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.073+0000 7f8dcfe00700 1 -- 172.21.15.16:0/112765683 learned_addr learned my addr 172.21.15.16:0/112765683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcf400700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 0x5624c499a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcfe00700 1 -- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 msgr2=0x5624c499a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 0x5624c499a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcfe00700 1 -- 172.21.15.16:0/112765683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 msgr2=0x5624c499ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c499ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c499ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcfe00700 1 -- 172.21.15.16:0/112765683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c49a80c0 con 0x5624c4996e70 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dcf400700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 0x5624c499a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.074+0000 7f8dc6a00700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c499ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.075+0000 7f8dbd600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:10:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.075+0000 7f8dcfe00700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8015fd0 tx=0x7f8dc8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7add518ac4e7fe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.075+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8023040 con 0x5624c4996e70 2024-05-27T07:10:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.076+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f8dc8006240 con 0x5624c4996e70 2024-05-27T07:10:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.076+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dc00429e0 con 0x5624c4996e70 2024-05-27T07:10:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.077+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8dc8005e50 con 0x5624c4996e70 2024-05-27T07:10:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.079+0000 7f8de5349b80 10 client.26176 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.079+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5624c49a9320 con 0x5624c4996e70 2024-05-27T07:10:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.080+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc00429e0 con 0x5624c4996e70 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.081+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8dc801e560 con 0x5624c4996e70 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 10 client.26176 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 10 client.26176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 20 client.26176 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c4a1b3e0 0x5624c4a1d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 4 client.26176 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:10:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.082+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624c48c0570 con 0x5624c4a1b3e0 2024-05-27T07:10:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.083+0000 7f8dcf400700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c4a1b3e0 0x5624c4a1d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.085+0000 7f8dbd600700 10 client.26176 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:10:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.086+0000 7f8dcf400700 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c4a1b3e0 0x5624c4a1d800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ca8fd3d42700b54 server_cookie=e1402f07f4a227da in_seq=0 out_seq=0 2024-05-27T07:10:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.086+0000 7f8dbd600700 1 -- 172.21.15.16:0/112765683 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624c48c0570 con 0x5624c4a1b3e0 2024-05-27T07:10:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.086+0000 7f8dbd600700 10 client.26176 handle_command_reply: tid=0 2024-05-27T07:10:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.086+0000 7f8de5349b80 1 client.26176 shutdown 2024-05-27T07:10:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 msgr2=0x5624c499b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8015fd0 tx=0x7f8dc8014000 comp rx=0 tx=0).stop 2024-05-27T07:10:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 shutdown_connections 2024-05-27T07:10:31.081 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:10:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624c4996e70 0x5624c499b0b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x5624c4a1b3e0 0x5624c4a1d800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8dbd600700 0 client.26176 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624c4817e40 0x5624c499a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 --2- 172.21.15.16:0/112765683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624c49039f0 0x5624c499ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.088+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 >> 172.21.15.16:0/112765683 conn(0x5624c48b1390 msgr2=0x5624c49a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.089+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 shutdown_connections 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.089+0000 7f8de5349b80 1 -- 172.21.15.16:0/112765683 wait complete. 2024-05-27T07:10:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.089+0000 7f8de5349b80 20 client.26176 trim_cache size 0 max 16384 2024-05-27T07:10:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.092+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 msgr2=0x7f8db40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.092+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 0x7f8db40b4f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001ae70 tx=0x7f8dc0018f90 comp rx=0 tx=0).stop 2024-05-27T07:10:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.092+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 msgr2=0x7f8dd0119260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.092+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd0119260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db800b730 comp rx=0 tx=0).stop 2024-05-27T07:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 shutdown_connections 2024-05-27T07:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f8db40b2a70 0x7f8db40b4f80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dd011d920 0x7f8dd01197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd010a920 0x7f8dd011d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 --2- 172.21.15.16:0/283164000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dd010b270 0x7f8dd0119260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 >> 172.21.15.16:0/283164000 conn(0x7f8dd006f4c0 msgr2=0x7f8dd011df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.093+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 shutdown_connections 2024-05-27T07:10:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:31.094+0000 7f8dd6800700 1 -- 172.21.15.16:0/283164000 wait complete. 2024-05-27T07:10:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:31 smithi164 ceph-mon[35327]: pgmap v299: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 413 op/s 2024-05-27T07:10:32.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:31 smithi016 ceph-mon[31415]: pgmap v299: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 413 op/s 2024-05-27T07:10:32.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:31 smithi069 ceph-mon[35276]: pgmap v299: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 413 op/s 2024-05-27T07:10:32.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:10:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:10:33.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:33 smithi016 ceph-mon[31415]: pgmap v300: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 518 op/s 2024-05-27T07:10:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:33 smithi164 ceph-mon[35327]: pgmap v300: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 518 op/s 2024-05-27T07:10:34.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:33 smithi069 ceph-mon[35276]: pgmap v300: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 518 op/s 2024-05-27T07:10:35.990 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:35 smithi016 ceph-mon[31415]: pgmap v301: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:35 smithi164 ceph-mon[35327]: pgmap v301: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:36.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:35 smithi069 ceph-mon[35276]: pgmap v301: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:37.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:37 smithi016 ceph-mon[31415]: pgmap v302: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:38.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:37 smithi164 ceph-mon[35327]: pgmap v302: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:38.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:37 smithi069 ceph-mon[35276]: pgmap v302: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 26 MiB/s wr, 525 op/s 2024-05-27T07:10:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:39 smithi164 ceph-mon[35327]: pgmap v303: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 501 op/s 2024-05-27T07:10:40.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:39 smithi016 ceph-mon[31415]: pgmap v303: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 501 op/s 2024-05-27T07:10:40.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:39 smithi069 ceph-mon[35276]: pgmap v303: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 501 op/s 2024-05-27T07:10:42.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 -- 172.21.15.16:0/23341541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4110280 msgr2=0x7f3fc4117be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/23341541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4110280 0x7f3fc4117be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b280 tx=0x7f3fbc0089d0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 -- 172.21.15.16:0/23341541 shutdown_connections 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/23341541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4110280 0x7f3fc4117be0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/23341541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc410b000 0x7f3fc410b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/23341541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc410fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.053+0000 7f3fc9e00700 1 -- 172.21.15.16:0/23341541 >> 172.21.15.16:0/23341541 conn(0x7f3fc406f4c0 msgr2=0x7f3fc406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.047 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:41 smithi016 ceph-mon[31415]: pgmap v304: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 464 op/s 2024-05-27T07:10:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.054+0000 7f3fc9e00700 1 -- 172.21.15.16:0/23341541 shutdown_connections 2024-05-27T07:10:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.054+0000 7f3fc9e00700 1 -- 172.21.15.16:0/23341541 wait complete. 2024-05-27T07:10:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.056+0000 7f3fc9e00700 1 Processor -- start 2024-05-27T07:10:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.056+0000 7f3fc9e00700 1 -- start start 2024-05-27T07:10:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.057+0000 7f3fc9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc410b000 0x7f3fc41a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.057+0000 7f3fc9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc4110280 0x7f3fc41a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc9e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc411a950 con 0x7f3fc4110280 2024-05-27T07:10:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36352/0 (socket says 172.21.15.16:36352) 2024-05-27T07:10:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc2000700 1 -- 172.21.15.16:0/255356128 learned_addr learned my addr 172.21.15.16:0/255356128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc411a7d0 con 0x7f3fc410b000 2024-05-27T07:10:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.058+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc411aad0 con 0x7f3fc410f930 2024-05-27T07:10:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc3400700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc4110280 0x7f3fc41a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc2000700 1 -- 172.21.15.16:0/255356128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc410b000 msgr2=0x7f3fc41a5730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:10:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc2000700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc410b000 0x7f3fc41a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc2000700 1 -- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc4110280 msgr2=0x7f3fc41a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc2000700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc4110280 0x7f3fc41a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.059+0000 7f3fc2000700 1 -- 172.21.15.16:0/255356128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a5200 con 0x7f3fc410f930 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.060+0000 7f3fc2000700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005950 tx=0x7f3fb4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.060+0000 7f3fc8a00700 1 -- 172.21.15.16:0/255356128 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4004830 con 0x7f3fc410f930 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.060+0000 7f3fc8a00700 1 -- 172.21.15.16:0/255356128 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb4005020 con 0x7f3fc410f930 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.060+0000 7f3fc8a00700 1 -- 172.21.15.16:0/255356128 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4013810 con 0x7f3fc410f930 2024-05-27T07:10:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.060+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41aa500 con 0x7f3fc410f930 2024-05-27T07:10:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.061+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41aa9f0 con 0x7f3fc410f930 2024-05-27T07:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.067+0000 7f3fc8a00700 1 -- 172.21.15.16:0/255356128 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3fb401b020 con 0x7f3fc410f930 2024-05-27T07:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.070+0000 7f3fc8a00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 0x7f3fa80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.070+0000 7f3fc8a00700 1 -- 172.21.15.16:0/255356128 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f3fb40e1f00 con 0x7f3fc410f930 2024-05-27T07:10:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.073+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 0x7f3fa80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.079+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 0x7f3fa80b50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000ba10 tx=0x7f3fb000b3f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:41 smithi164 ceph-mon[35327]: pgmap v304: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 464 op/s 2024-05-27T07:10:42.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:41 smithi069 ceph-mon[35276]: pgmap v304: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 464 op/s 2024-05-27T07:10:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.173+0000 7f3fd890eb80 1 Processor -- start 2024-05-27T07:10:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.173+0000 7f3fd890eb80 1 -- start start 2024-05-27T07:10:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b30c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b30d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e1b23ae40 con 0x556e1b312d30 2024-05-27T07:10:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e1b22a9e0 con 0x556e1b18de40 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.174+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e1b22ea20 con 0x556e1b30cdb0 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b30c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60168/0 (socket says 172.21.15.16:60168) 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc2a00700 1 -- 172.21.15.16:0/1568713026 learned_addr learned my addr 172.21.15.16:0/1568713026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc3400700 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b30d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.175+0000 7f3fc2a00700 1 -- 172.21.15.16:0/1568713026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 msgr2=0x556e1b30d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.176+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b30d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.176+0000 7f3fc2a00700 1 -- 172.21.15.16:0/1568713026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 msgr2=0x556e1b30c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.176+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b30c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.176+0000 7f3fc2a00700 1 -- 172.21.15.16:0/1568713026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1b1f3420 con 0x556e1b18de40 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.176+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000dbd0 tx=0x7f3fb0010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ddd9b1ae89781c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.177+0000 7f3fb8c00700 1 -- 172.21.15.16:0/1568713026 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0010af0 con 0x556e1b18de40 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.177+0000 7f3fb8c00700 1 -- 172.21.15.16:0/1568713026 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb0022070 con 0x556e1b18de40 2024-05-27T07:10:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.177+0000 7f3fb8c00700 1 -- 172.21.15.16:0/1568713026 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000f3c0 con 0x556e1b18de40 2024-05-27T07:10:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.179+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1568713026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 msgr2=0x556e1b3127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.179+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000dbd0 tx=0x7f3fb0010000 comp rx=0 tx=0).stop 2024-05-27T07:10:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.180+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1568713026 shutdown_connections 2024-05-27T07:10:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.180+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b30d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.180+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3127f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.180+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1568713026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b30c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.180+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1568713026 >> 172.21.15.16:0/1568713026 conn(0x556e1b1edfc0 msgr2=0x556e1b3078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.181+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1568713026 shutdown_connections 2024-05-27T07:10:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.181+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1568713026 wait complete. 2024-05-27T07:10:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.183+0000 7f3fd890eb80 1 -- start start 2024-05-27T07:10:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.183+0000 7f3fd890eb80 1 Processor -- start 2024-05-27T07:10:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b316a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b316f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e1b130fa0 con 0x556e1b312d30 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e1b1cb760 con 0x556e1b18de40 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e1b3197a0 con 0x556e1b30cdb0 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.185+0000 7f3fd890eb80 10 client.0 _ll_register_callbacks cb 0x556e1b1d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.187+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.187+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3164f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60180/0 (socket says 172.21.15.16:60180) 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.187+0000 7f3fc2a00700 1 -- 172.21.15.16:0/232348508 learned_addr learned my addr 172.21.15.16:0/232348508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.187+0000 7f3fc2000700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b316a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.188+0000 7f3fc2000700 1 -- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 msgr2=0x556e1b3164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.188+0000 7f3fc2000700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.188+0000 7f3fc2000700 1 -- 172.21.15.16:0/232348508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 msgr2=0x556e1b316f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.188+0000 7f3fc2000700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b316f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.188+0000 7f3fc2000700 1 -- 172.21.15.16:0/232348508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1b17a8d0 con 0x556e1b30cdb0 2024-05-27T07:10:42.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.191+0000 7f3fc2000700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b316a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40df210 tx=0x7f3fb40df770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205aaf804205df7c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.191+0000 7f3fad600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:10:42.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.192+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40ad150 con 0x556e1b30cdb0 2024-05-27T07:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.193+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f3fb40a9210 con 0x556e1b30cdb0 2024-05-27T07:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.193+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb0022070 con 0x556e1b30cdb0 2024-05-27T07:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.193+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3fb40b1c00 con 0x556e1b30cdb0 2024-05-27T07:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.194+0000 7f3fd890eb80 10 client.26188 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.194+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e1b1fb060 con 0x556e1b30cdb0 2024-05-27T07:10:42.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.195+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb0022070 con 0x556e1b30cdb0 2024-05-27T07:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.195+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3fb40c1070 con 0x556e1b30cdb0 2024-05-27T07:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.196+0000 7f3fd890eb80 10 client.26188 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.196+0000 7f3fd890eb80 10 client.26188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.196+0000 7f3fd890eb80 20 client.26188 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.196+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b3266f0 0x556e1b328b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.197+0000 7f3fd890eb80 4 client.26188 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.197+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1b23cd90 con 0x556e1b3266f0 2024-05-27T07:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.200+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b3266f0 0x556e1b328b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.201+0000 7f3fad600700 10 client.26188 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.201+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b3266f0 0x556e1b328b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1111009d223ad78a server_cookie=dfcb84ddd5d57838 in_seq=0 out_seq=0 2024-05-27T07:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.202+0000 7f3fad600700 1 -- 172.21.15.16:0/232348508 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e1b23cd90 con 0x556e1b3266f0 2024-05-27T07:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.202+0000 7f3fad600700 10 client.26188 handle_command_reply: tid=0 2024-05-27T07:10:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.202+0000 7f3fd890eb80 1 client.26188 shutdown 2024-05-27T07:10:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 msgr2=0x556e1b316a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b316a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40df210 tx=0x7f3fb40df770 comp rx=0 tx=0).stop 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 shutdown_connections 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b3266f0 0x556e1b328b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b316f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b3164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/232348508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b30cdb0 0x556e1b316a30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 >> 172.21.15.16:0/232348508 conn(0x556e1b1edfc0 msgr2=0x556e1b30d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.203+0000 7f3fad600700 0 client.26188 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:10:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.204+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 shutdown_connections 2024-05-27T07:10:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.204+0000 7f3fd890eb80 1 -- 172.21.15.16:0/232348508 wait complete. 2024-05-27T07:10:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.205+0000 7f3fd890eb80 20 client.26188 trim_cache size 0 max 16384 2024-05-27T07:10:42.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.237+0000 7f3fd890eb80 1 Processor -- start 2024-05-27T07:10:42.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.238+0000 7f3fd890eb80 1 -- start start 2024-05-27T07:10:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b18de40 0x556e1b30c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b310850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41bf5b0 con 0x556e1b312d30 2024-05-27T07:10:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41c10f0 con 0x556e1b30cdb0 2024-05-27T07:10:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41bfc80 con 0x556e1b18de40 2024-05-27T07:10:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.239+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b18de40 0x556e1b30c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.240+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.240+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60190/0 (socket says 172.21.15.16:60190) 2024-05-27T07:10:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.240+0000 7f3fc2000700 1 -- 172.21.15.16:0/3158623414 learned_addr learned my addr 172.21.15.16:0/3158623414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.240+0000 7f3fc3400700 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b310850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.241+0000 7f3fc2000700 1 -- 172.21.15.16:0/3158623414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b18de40 msgr2=0x556e1b30c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.241+0000 7f3fc2000700 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b18de40 0x556e1b30c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.241+0000 7f3fc2000700 1 -- 172.21.15.16:0/3158623414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 msgr2=0x556e1b310850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.241+0000 7f3fc2000700 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b310850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.241+0000 7f3fc2000700 1 -- 172.21.15.16:0/3158623414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1b3172e0 con 0x556e1b30cdb0 2024-05-27T07:10:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.242+0000 7f3fc2000700 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40a89e0 tx=0x7f3fb40235b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06888b31c2e7298 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.242+0000 7f3fafe00700 1 -- 172.21.15.16:0/3158623414 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40a8b70 con 0x556e1b30cdb0 2024-05-27T07:10:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.243+0000 7f3fafe00700 1 -- 172.21.15.16:0/3158623414 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb40b1e50 con 0x556e1b30cdb0 2024-05-27T07:10:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.243+0000 7f3fafe00700 1 -- 172.21.15.16:0/3158623414 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40a80f0 con 0x556e1b30cdb0 2024-05-27T07:10:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.244+0000 7f3fd890eb80 1 -- 172.21.15.16:0/3158623414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 msgr2=0x556e1b322800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.244+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40a89e0 tx=0x7f3fb40235b0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 -- 172.21.15.16:0/3158623414 shutdown_connections 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b312d30 0x556e1b310850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b18de40 0x556e1b30c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/3158623414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b30cdb0 0x556e1b322800 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 -- 172.21.15.16:0/3158623414 >> 172.21.15.16:0/3158623414 conn(0x556e1b1edfc0 msgr2=0x556e1b310140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 -- 172.21.15.16:0/3158623414 shutdown_connections 2024-05-27T07:10:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.245+0000 7f3fd890eb80 1 -- 172.21.15.16:0/3158623414 wait complete. 2024-05-27T07:10:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.246+0000 7f3fd890eb80 1 -- start start 2024-05-27T07:10:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.246+0000 7f3fd890eb80 1 Processor -- start 2024-05-27T07:10:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.247+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b31e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.247+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.247+0000 7f3fd890eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 0x556e1b32d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46474/0 (socket says 172.21.15.16:46474) 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b31e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b31e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:60202/0 (socket says 172.21.15.16:60202) 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41c0a20 con 0x556e1b30cdb0 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41c0350 con 0x556e1b18de40 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fd890eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41bd570 con 0x556e1b312d30 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fd890eb80 10 client.0 _ll_register_callbacks cb 0x556e1b1d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.248+0000 7f3fc2000700 1 -- 172.21.15.16:0/1511555236 learned_addr learned my addr 172.21.15.16:0/1511555236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.249+0000 7f3fc3400700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 0x556e1b32d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc3400700 1 -- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 msgr2=0x556e1b31e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc3400700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b31e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc3400700 1 -- 172.21.15.16:0/1511555236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 msgr2=0x556e1b31ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc3400700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc3400700 1 -- 172.21.15.16:0/1511555236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1b32d940 con 0x556e1b312d30 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.250+0000 7f3fc2000700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.251+0000 7f3fc3400700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 0x556e1b32d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b1a0 tx=0x7f3fbc0125f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d11e803111ac66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.251+0000 7f3fad600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.251+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc018040 con 0x556e1b312d30 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.251+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f3fbc012690 con 0x556e1b312d30 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.251+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb40b1e50 con 0x556e1b312d30 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.252+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3fbc02f050 con 0x556e1b312d30 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.252+0000 7f3fd890eb80 10 client.26194 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.252+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e1b390980 con 0x556e1b312d30 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fbc012e90 con 0x556e1b312d30 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3fbc01e040 con 0x556e1b312d30 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fd890eb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fd890eb80 10 client.26194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fd890eb80 20 client.26194 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.254+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b391130 0x556e1b393550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.255+0000 7f3fd890eb80 4 client.26194 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:10:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.255+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3fb40b1e50 con 0x556e1b391130 2024-05-27T07:10:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.255+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b391130 0x556e1b393550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.256+0000 7f3fc2a00700 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b391130 0x556e1b393550 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20a86cb5a050f1c3 server_cookie=f82aef3bb4d8ac0a in_seq=0 out_seq=0 2024-05-27T07:10:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.256+0000 7f3fad600700 10 client.26194 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:10:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.257+0000 7f3fad600700 1 -- 172.21.15.16:0/1511555236 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3fb40b1e50 con 0x556e1b391130 2024-05-27T07:10:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.257+0000 7f3fad600700 10 client.26194 handle_command_reply: tid=0 2024-05-27T07:10:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.257+0000 7f3fd890eb80 1 client.26194 shutdown 2024-05-27T07:10:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.257+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 msgr2=0x556e1b32d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.257+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 0x556e1b32d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b1a0 tx=0x7f3fbc0125f0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 shutdown_connections 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x556e1b391130 0x556e1b393550 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e1b312d30 0x556e1b32d380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e1b18de40 0x556e1b31e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 --2- 172.21.15.16:0/1511555236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1b30cdb0 0x556e1b31ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.258+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 >> 172.21.15.16:0/1511555236 conn(0x556e1b1edfc0 msgr2=0x556e1b3108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.260+0000 7f3fad600700 0 client.26194 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:10:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.262+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 shutdown_connections 2024-05-27T07:10:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.262+0000 7f3fd890eb80 1 -- 172.21.15.16:0/1511555236 wait complete. 2024-05-27T07:10:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.262+0000 7f3fd890eb80 20 client.26194 trim_cache size 0 max 16384 2024-05-27T07:10:42.254 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:10:42.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 msgr2=0x7f3fa80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 0x7f3fa80b50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000ba10 tx=0x7f3fb000b3f0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 msgr2=0x7f3fc41a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005950 tx=0x7f3fb4009b60 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 shutdown_connections 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f3fa80b2bc0 0x7f3fa80b50d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc4110280 0x7f3fc41a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc410b000 0x7f3fc41a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 --2- 172.21.15.16:0/255356128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc410f930 0x7f3fc41a5cd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.264+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 >> 172.21.15.16:0/255356128 conn(0x7f3fc406f4c0 msgr2=0x7f3fc410da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:42.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.265+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 shutdown_connections 2024-05-27T07:10:42.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:42.265+0000 7f3fc9e00700 1 -- 172.21.15.16:0/255356128 wait complete. 2024-05-27T07:10:43.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:42 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:10:43.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:42 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:10:43.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:42 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:10:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:43 smithi164 ceph-mon[35327]: pgmap v305: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 581 op/s 2024-05-27T07:10:44.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:43 smithi016 ceph-mon[31415]: pgmap v305: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 581 op/s 2024-05-27T07:10:44.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:43 smithi069 ceph-mon[35276]: pgmap v305: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 581 op/s 2024-05-27T07:10:44.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:10:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:10:46.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:45 smithi164 ceph-mon[35327]: pgmap v306: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:10:46.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:45 smithi016 ceph-mon[31415]: pgmap v306: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:10:46.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:45 smithi069 ceph-mon[35276]: pgmap v306: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:10:47.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:46 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:10:47.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:46 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:10:47.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:46 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:10:47.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:46 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:10:47.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:46 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:10:47.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:46 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:10:47.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:46 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:10:47.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:46 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:10:47.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:46 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:10:47.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:47 smithi016 ceph-mon[31415]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 509 op/s 2024-05-27T07:10:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:47 smithi164 ceph-mon[35327]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 509 op/s 2024-05-27T07:10:48.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:47 smithi069 ceph-mon[35276]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 509 op/s 2024-05-27T07:10:50.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:49 smithi164 ceph-mon[35327]: pgmap v308: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 24 MiB/s wr, 508 op/s 2024-05-27T07:10:50.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:49 smithi016 ceph-mon[31415]: pgmap v308: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 24 MiB/s wr, 508 op/s 2024-05-27T07:10:50.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:49 smithi069 ceph-mon[35276]: pgmap v308: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 24 MiB/s wr, 508 op/s 2024-05-27T07:10:52.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:51 smithi016 ceph-mon[31415]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 22 MiB/s wr, 465 op/s 2024-05-27T07:10:52.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:51 smithi069 ceph-mon[35276]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 22 MiB/s wr, 465 op/s 2024-05-27T07:10:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:51 smithi164 ceph-mon[35327]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 22 MiB/s wr, 465 op/s 2024-05-27T07:10:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.859+0000 7f43be400700 1 -- 172.21.15.16:0/774605113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 msgr2=0x7f43b8108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.859+0000 7f43be400700 1 --2- 172.21.15.16:0/774605113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b8108010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009990 tx=0x7f43ac009ca0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 -- 172.21.15.16:0/774605113 shutdown_connections 2024-05-27T07:10:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 --2- 172.21.15.16:0/774605113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 --2- 172.21.15.16:0/774605113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 --2- 172.21.15.16:0/774605113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b8108010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 -- 172.21.15.16:0/774605113 >> 172.21.15.16:0/774605113 conn(0x7f43b8078ab0 msgr2=0x7f43b8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 -- 172.21.15.16:0/774605113 shutdown_connections 2024-05-27T07:10:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.863+0000 7f43be400700 1 -- 172.21.15.16:0/774605113 wait complete. 2024-05-27T07:10:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.865+0000 7f43be400700 1 Processor -- start 2024-05-27T07:10:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.867+0000 7f43be400700 1 -- start start 2024-05-27T07:10:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.870+0000 7f43be400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.870+0000 7f43be400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b80727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.870+0000 7f43be400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b8072d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.870+0000 7f43bd000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.870+0000 7f43bd000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:37420/0 (socket says 172.21.15.16:37420) 2024-05-27T07:10:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.871+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b80727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.871+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b80727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37710/0 (socket says 172.21.15.16:37710) 2024-05-27T07:10:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.872+0000 7f43bd000700 1 -- 172.21.15.16:0/1011624741 learned_addr learned my addr 172.21.15.16:0/1011624741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.872+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43b806c3f0 con 0x7f43b8108550 2024-05-27T07:10:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.872+0000 7f43bda00700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b8072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.872+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43b806c270 con 0x7f43b8107c00 2024-05-27T07:10:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.873+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43b806c570 con 0x7f43b81032d0 2024-05-27T07:10:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.878+0000 7f43bd000700 1 -- 172.21.15.16:0/1011624741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 msgr2=0x7f43b80727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.878+0000 7f43bd000700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b80727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.878+0000 7f43bd000700 1 -- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 msgr2=0x7f43b8072d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.878+0000 7f43bd000700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b8072d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.879+0000 7f43bd000700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b8071300 con 0x7f43b81032d0 2024-05-27T07:10:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.879+0000 7f43bda00700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b8072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.880+0000 7f43bd000700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c950 tx=0x7f43a400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.880+0000 7f43b5600700 1 -- 172.21.15.16:0/1011624741 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a40078d0 con 0x7f43b81032d0 2024-05-27T07:10:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.881+0000 7f43b5600700 1 -- 172.21.15.16:0/1011624741 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43a400f450 con 0x7f43b81032d0 2024-05-27T07:10:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.881+0000 7f43b5600700 1 -- 172.21.15.16:0/1011624741 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4018660 con 0x7f43b81032d0 2024-05-27T07:10:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.881+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b80715f0 con 0x7f43b81032d0 2024-05-27T07:10:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.881+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b8071ae0 con 0x7f43b81032d0 2024-05-27T07:10:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.889+0000 7f43b5600700 1 -- 172.21.15.16:0/1011624741 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f43a4018800 con 0x7f43b81032d0 2024-05-27T07:10:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.892+0000 7f43b5600700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 0x7f43a00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.892+0000 7f43b5600700 1 -- 172.21.15.16:0/1011624741 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f43a40e3680 con 0x7f43b81032d0 2024-05-27T07:10:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.892+0000 7f43b7e00700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 0x7f43a00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.933+0000 7f43b7e00700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 0x7f43a00b5030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f43b8113d60 tx=0x7f43ac005fd0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:52.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.937+0000 7f43ce5fdb80 1 Processor -- start 2024-05-27T07:10:52.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.938+0000 7f43ce5fdb80 1 -- start start 2024-05-27T07:10:52.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.938+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.939+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.939+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.939+0000 7f43ce5fdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558a45981e40 con 0x558a458d4e40 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.939+0000 7f43ce5fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a459719e0 con 0x558a45a537b0 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.939+0000 7f43ce5fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a45975a20 con 0x558a45a533a0 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.940+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.940+0000 7f43bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.940+0000 7f43bda00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.940+0000 7f43bd000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55778/0 (socket says 172.21.15.16:55778) 2024-05-27T07:10:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.941+0000 7f43bd000700 1 -- 172.21.15.16:0/3799321001 learned_addr learned my addr 172.21.15.16:0/3799321001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.942+0000 7f43bda00700 1 -- 172.21.15.16:0/3799321001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 msgr2=0x558a45a5db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.940+0000 7f43b7e00700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:37426/0 (socket says 172.21.15.16:37426) 2024-05-27T07:10:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.942+0000 7f43bda00700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.942+0000 7f43bda00700 1 -- 172.21.15.16:0/3799321001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 msgr2=0x558a45a52e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.942+0000 7f43bda00700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.943+0000 7f43bda00700 1 -- 172.21.15.16:0/3799321001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4593a420 con 0x558a45a537b0 2024-05-27T07:10:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.943+0000 7f43bd000700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.946+0000 7f43b7e00700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:10:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.947+0000 7f43bda00700 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a60060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a740 tx=0x7f43b0009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e57911194a0c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:52.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.947+0000 7f43aa000700 1 -- 172.21.15.16:0/3799321001 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0013790 con 0x558a45a537b0 2024-05-27T07:10:52.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.947+0000 7f43aa000700 1 -- 172.21.15.16:0/3799321001 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43b0019070 con 0x558a45a537b0 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.948+0000 7f43aa000700 1 -- 172.21.15.16:0/3799321001 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0014c80 con 0x558a45a537b0 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/3799321001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 msgr2=0x558a45a60060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a60060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a740 tx=0x7f43b0009650 comp rx=0 tx=0).stop 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/3799321001 shutdown_connections 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a60060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a458d4e40 0x558a45a52e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/3799321001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a533a0 0x558a45a5db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.949+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/3799321001 >> 172.21.15.16:0/3799321001 conn(0x558a45934fc0 msgr2=0x558a45a4ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.951+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/3799321001 shutdown_connections 2024-05-27T07:10:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.951+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/3799321001 wait complete. 2024-05-27T07:10:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.952+0000 7f43ce5fdb80 1 -- start start 2024-05-27T07:10:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.952+0000 7f43ce5fdb80 1 Processor -- start 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.954+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.954+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.954+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a537b0 0x558a45a664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.955+0000 7f43bd000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.955+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.955+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37742/0 (socket says 172.21.15.16:37742) 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.955+0000 7f43b7e00700 1 -- 172.21.15.16:0/2229342686 learned_addr learned my addr 172.21.15.16:0/2229342686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.955+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558a45877fa0 con 0x558a45a537b0 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.956+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a45912760 con 0x558a45a533a0 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.956+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a45a609d0 con 0x558a458d4e40 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.956+0000 7f43ce5fdb80 10 client.0 _ll_register_callbacks cb 0x558a4591bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.956+0000 7f43bda00700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a537b0 0x558a45a664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.957+0000 7f43b7e00700 1 -- 172.21.15.16:0/2229342686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 msgr2=0x558a45a6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.957+0000 7f43b7e00700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.957+0000 7f43b7e00700 1 -- 172.21.15.16:0/2229342686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a537b0 msgr2=0x558a45a664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.957+0000 7f43b7e00700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a537b0 0x558a45a664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.957+0000 7f43b7e00700 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a458c18d0 con 0x558a45a533a0 2024-05-27T07:10:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.958+0000 7f439ea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:10:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.958+0000 7f43b7e00700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009990 tx=0x7f43ac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d901327f57334670 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.958+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac033060 con 0x558a45a533a0 2024-05-27T07:10:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.959+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f43ac003ba0 con 0x558a45a533a0 2024-05-27T07:10:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.962+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b0019070 con 0x558a45a533a0 2024-05-27T07:10:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.962+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43ac003da0 con 0x558a45a533a0 2024-05-27T07:10:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.962+0000 7f43ce5fdb80 10 client.36269 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.962+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558a45942060 con 0x558a45a533a0 2024-05-27T07:10:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.963+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43b0019070 con 0x558a45a533a0 2024-05-27T07:10:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.963+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f43ac022070 con 0x558a45a533a0 2024-05-27T07:10:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.963+0000 7f43ce5fdb80 10 client.36269 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.963+0000 7f43ce5fdb80 10 client.36269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:10:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.963+0000 7f43ce5fdb80 20 client.36269 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.964+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45a76070 0x558a45a784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.964+0000 7f43ce5fdb80 4 client.36269 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:10:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.964+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a45983d90 con 0x558a45a76070 2024-05-27T07:10:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.965+0000 7f43bd000700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45a76070 0x558a45a784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.966+0000 7f439ea00700 10 client.36269 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:10:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.966+0000 7f43bd000700 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45a76070 0x558a45a784c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c6c352fb67f5f18 server_cookie=83134bb2c3528b17 in_seq=0 out_seq=0 2024-05-27T07:10:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.969+0000 7f439ea00700 1 -- 172.21.15.16:0/2229342686 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558a45983d90 con 0x558a45a76070 2024-05-27T07:10:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.970+0000 7f439ea00700 10 client.36269 handle_command_reply: tid=0 2024-05-27T07:10:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.970+0000 7f43ce5fdb80 1 client.36269 shutdown 2024-05-27T07:10:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.971+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 msgr2=0x558a45a6c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.971+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009990 tx=0x7f43ac009ca0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 shutdown_connections 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45a76070 0x558a45a784c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a537b0 0x558a45a664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/2229342686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a533a0 0x558a45a6c5e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 >> 172.21.15.16:0/2229342686 conn(0x558a45934fc0 msgr2=0x558a45a4dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.972+0000 7f439ea00700 0 client.36269 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:10:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.973+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 shutdown_connections 2024-05-27T07:10:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.973+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/2229342686 wait complete. 2024-05-27T07:10:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:52.973+0000 7f43ce5fdb80 20 client.36269 trim_cache size 0 max 16384 2024-05-27T07:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.015+0000 7f43ce5fdb80 1 Processor -- start 2024-05-27T07:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.015+0000 7f43ce5fdb80 1 -- start start 2024-05-27T07:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.016+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 0x558a45a4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.016+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.017+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.017+0000 7f43bd000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 0x558a45a4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.017+0000 7f43bd000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 0x558a45a4e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37752/0 (socket says 172.21.15.16:37752) 2024-05-27T07:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.018+0000 7f43bda00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.018+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.018+0000 7f43bda00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:37452/0 (socket says 172.21.15.16:37452) 2024-05-27T07:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.018+0000 7f43bda00700 1 -- 172.21.15.16:0/486431355 learned_addr learned my addr 172.21.15.16:0/486431355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.019+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f439800ee30 con 0x558a45a533a0 2024-05-27T07:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.019+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4398010970 con 0x558a458d4e40 2024-05-27T07:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.019+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f439800f500 con 0x558a45a537b0 2024-05-27T07:10:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.020+0000 7f43bda00700 1 -- 172.21.15.16:0/486431355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 msgr2=0x558a45a4e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.020+0000 7f43bda00700 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 0x558a45a4e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.020+0000 7f43bda00700 1 -- 172.21.15.16:0/486431355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 msgr2=0x558a45a4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.020+0000 7f43bda00700 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.020+0000 7f43bda00700 1 -- 172.21.15.16:0/486431355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a45a546e0 con 0x558a45a537b0 2024-05-27T07:10:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.021+0000 7f43b7e00700 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.024+0000 7f43bda00700 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558a45a53c60 tx=0x7f43b0013720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c15a0fe8fa9be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.025+0000 7f43a9600700 1 -- 172.21.15.16:0/486431355 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0013ad0 con 0x558a45a537b0 2024-05-27T07:10:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.025+0000 7f43a9600700 1 -- 172.21.15.16:0/486431355 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43b0021070 con 0x558a45a537b0 2024-05-27T07:10:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.025+0000 7f43a9600700 1 -- 172.21.15.16:0/486431355 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0014420 con 0x558a45a537b0 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 msgr2=0x558a45a78080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558a45a53c60 tx=0x7f43b0013720 comp rx=0 tx=0).stop 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 shutdown_connections 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a45a537b0 0x558a45a78080 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a458d4e40 0x558a45a4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/486431355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 >> 172.21.15.16:0/486431355 conn(0x558a45934fc0 msgr2=0x558a45a66280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.027+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 shutdown_connections 2024-05-27T07:10:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.028+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/486431355 wait complete. 2024-05-27T07:10:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.028+0000 7f43ce5fdb80 1 -- start start 2024-05-27T07:10:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.028+0000 7f43ce5fdb80 1 Processor -- start 2024-05-27T07:10:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.031+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a72480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.031+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a6f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.031+0000 7f43ce5fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.032+0000 7f43bda00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.032+0000 7f43bda00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:37768/0 (socket says 172.21.15.16:37768) 2024-05-27T07:10:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.032+0000 7f43bda00700 1 -- 172.21.15.16:0/353236076 learned_addr learned my addr 172.21.15.16:0/353236076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.033+0000 7f43b7e00700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a6f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.033+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43980102a0 con 0x558a45a533a0 2024-05-27T07:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.033+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f439800fbd0 con 0x558a45a537b0 2024-05-27T07:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.033+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f439800cdf0 con 0x558a458d4e40 2024-05-27T07:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43bda00700 1 -- 172.21.15.16:0/353236076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 msgr2=0x558a45a72480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43bda00700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a72480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43bda00700 1 -- 172.21.15.16:0/353236076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 msgr2=0x558a45a6f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43bda00700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a6f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43bda00700 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a45a70470 con 0x558a45a537b0 2024-05-27T07:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43ce5fdb80 10 client.0 _ll_register_callbacks cb 0x558a4591bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.034+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b0021070 con 0x558a45a537b0 2024-05-27T07:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.035+0000 7f43b7e00700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a6f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:10:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.036+0000 7f439ea00700 10 client.36275 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:10:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.036+0000 7f43bda00700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a710 tx=0x7f43b00136b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e00f5da6a16a38f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.037+0000 7f439ea00700 1 -- 172.21.15.16:0/353236076 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0036050 con 0x558a45a537b0 2024-05-27T07:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.037+0000 7f439ea00700 1 -- 172.21.15.16:0/353236076 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f43b002d020 con 0x558a45a537b0 2024-05-27T07:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.037+0000 7f439ea00700 1 -- 172.21.15.16:0/353236076 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43b0036050 con 0x558a45a537b0 2024-05-27T07:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.038+0000 7f43ce5fdb80 10 client.36275 fetch_fsmap learned FSMap version 31 2024-05-27T07:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.038+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x558a45ae0820 con 0x558a45a537b0 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.039+0000 7f439ea00700 1 -- 172.21.15.16:0/353236076 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f43b0026470 con 0x558a45a537b0 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.039+0000 7f43ce5fdb80 10 client.36275 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.039+0000 7f43ce5fdb80 10 client.36275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.039+0000 7f43ce5fdb80 20 client.36275 populate_metadata read hostname 'smithi016' 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.040+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45ae0b30 0x558a45ae2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.040+0000 7f43ce5fdb80 4 client.36275 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.040+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a4597d570 con 0x558a45ae0b30 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.040+0000 7f43bd000700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45ae0b30 0x558a45ae2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.041+0000 7f439ea00700 10 client.36275 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.041+0000 7f43bd000700 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45ae0b30 0x558a45ae2fd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70b1e7839ce6f25a server_cookie=f696b5eb66abe7a6 in_seq=0 out_seq=0 2024-05-27T07:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.042+0000 7f439ea00700 1 -- 172.21.15.16:0/353236076 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a4597d570 con 0x558a45ae0b30 2024-05-27T07:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.042+0000 7f439ea00700 10 client.36275 handle_command_reply: tid=0 2024-05-27T07:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.042+0000 7f43ce5fdb80 1 client.36275 shutdown 2024-05-27T07:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.042+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 msgr2=0x558a45a6fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.042+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a710 tx=0x7f43b00136b0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 shutdown_connections 2024-05-27T07:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x558a45ae0b30 0x558a45ae2fd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a45a537b0 0x558a45a6fd90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a458d4e40 0x558a45a72480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 --2- 172.21.15.16:0/353236076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a45a533a0 0x558a45a6f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 >> 172.21.15.16:0/353236076 conn(0x558a45934fc0 msgr2=0x558a45a4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f439ea00700 0 client.36275 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 shutdown_connections 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.043+0000 7f43ce5fdb80 1 -- 172.21.15.16:0/353236076 wait complete. 2024-05-27T07:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.044+0000 7f43ce5fdb80 20 client.36275 trim_cache size 0 max 16384 2024-05-27T07:10:53.038 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.047+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 msgr2=0x7f43a00b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.047+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 0x7f43a00b5030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f43b8113d60 tx=0x7f43ac005fd0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.047+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 msgr2=0x7f43b80722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.047+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c950 tx=0x7f43a400cc60 comp rx=0 tx=0).stop 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 shutdown_connections 2024-05-27T07:10:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f43a00b2b20 0x7f43a00b5030 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108550 0x7f43b8072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81032d0 0x7f43b80722b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 --2- 172.21.15.16:0/1011624741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8107c00 0x7f43b80727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 >> 172.21.15.16:0/1011624741 conn(0x7f43b8078ab0 msgr2=0x7f43b8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 shutdown_connections 2024-05-27T07:10:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:10:53.048+0000 7f43be400700 1 -- 172.21.15.16:0/1011624741 wait complete. 2024-05-27T07:10:53.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:53 smithi016 ceph-mon[31415]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 26 MiB/s wr, 555 op/s 2024-05-27T07:10:54.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:53 smithi069 ceph-mon[35276]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 26 MiB/s wr, 555 op/s 2024-05-27T07:10:54.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:10:54.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:53 smithi164 ceph-mon[35327]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 26 MiB/s wr, 555 op/s 2024-05-27T07:10:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:10:56.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:55 smithi016 ceph-mon[31415]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 487 op/s 2024-05-27T07:10:56.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:55 smithi069 ceph-mon[35276]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 487 op/s 2024-05-27T07:10:56.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:55 smithi164 ceph-mon[35327]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 487 op/s 2024-05-27T07:10:58.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:57 smithi016 ceph-mon[31415]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 24 MiB/s wr, 511 op/s 2024-05-27T07:10:58.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:57 smithi069 ceph-mon[35276]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 24 MiB/s wr, 511 op/s 2024-05-27T07:10:58.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:57 smithi164 ceph-mon[35327]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 24 MiB/s wr, 511 op/s 2024-05-27T07:11:00.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:10:59 smithi164 ceph-mon[35327]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 24 MiB/s wr, 495 op/s 2024-05-27T07:11:00.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:10:59 smithi016 ceph-mon[31415]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 24 MiB/s wr, 495 op/s 2024-05-27T07:11:00.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:10:59 smithi069 ceph-mon[35276]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 24 MiB/s wr, 495 op/s 2024-05-27T07:11:02.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:01 smithi069 ceph-mon[35276]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 454 op/s 2024-05-27T07:11:02.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:01 smithi164 ceph-mon[35327]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 454 op/s 2024-05-27T07:11:02.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:01 smithi016 ceph-mon[31415]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 454 op/s 2024-05-27T07:11:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.108+0000 7f6411200700 1 -- 172.21.15.16:0/2875156885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c10f930 msgr2=0x7f640c10fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.100 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:03 smithi016 ceph-mon[31415]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 27 MiB/s wr, 564 op/s 2024-05-27T07:11:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.108+0000 7f6411200700 1 --2- 172.21.15.16:0/2875156885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c10f930 0x7f640c10fd40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009990 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.109+0000 7f6411200700 1 -- 172.21.15.16:0/2875156885 shutdown_connections 2024-05-27T07:11:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.109+0000 7f6411200700 1 --2- 172.21.15.16:0/2875156885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c110280 0x7f640c117be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.111+0000 7f6411200700 1 --2- 172.21.15.16:0/2875156885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10b000 0x7f640c10b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.111+0000 7f6411200700 1 --2- 172.21.15.16:0/2875156885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c10f930 0x7f640c10fd40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.111+0000 7f6411200700 1 -- 172.21.15.16:0/2875156885 >> 172.21.15.16:0/2875156885 conn(0x7f640c06f4c0 msgr2=0x7f640c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.112+0000 7f6411200700 1 -- 172.21.15.16:0/2875156885 shutdown_connections 2024-05-27T07:11:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.113+0000 7f6411200700 1 -- 172.21.15.16:0/2875156885 wait complete. 2024-05-27T07:11:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.114+0000 7f6411200700 1 Processor -- start 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.114+0000 7f6411200700 1 -- start start 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 0x7f640c1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10f930 0x7f640c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f640c11a980 con 0x7f640c10f930 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f640c11a800 con 0x7f640c110280 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.115+0000 7f6411200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f640c11ab00 con 0x7f640c10b000 2024-05-27T07:11:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f640a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f640a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58748/0 (socket says 172.21.15.16:58748) 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f6409e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 0x7f640c1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f6409e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 0x7f640c1a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59360/0 (socket says 172.21.15.16:59360) 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f6409e00700 1 -- 172.21.15.16:0/3331658740 learned_addr learned my addr 172.21.15.16:0/3331658740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.116+0000 7f6409400700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10f930 0x7f640c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.117+0000 7f640a800700 1 -- 172.21.15.16:0/3331658740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 msgr2=0x7f640c1a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.117+0000 7f640a800700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 0x7f640c1a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.117+0000 7f640a800700 1 -- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10f930 msgr2=0x7f640c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.117+0000 7f640a800700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10f930 0x7f640c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.117+0000 7f640a800700 1 -- 172.21.15.16:0/3331658740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a6220 con 0x7f640c110280 2024-05-27T07:11:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.118+0000 7f640a800700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f64040049d0 tx=0x7f640400a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.119+0000 7f6402a00700 1 -- 172.21.15.16:0/3331658740 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404018020 con 0x7f640c110280 2024-05-27T07:11:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.119+0000 7f6402a00700 1 -- 172.21.15.16:0/3331658740 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6404004bb0 con 0x7f640c110280 2024-05-27T07:11:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.119+0000 7f6402a00700 1 -- 172.21.15.16:0/3331658740 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404009690 con 0x7f640c110280 2024-05-27T07:11:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.119+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a6510 con 0x7f640c110280 2024-05-27T07:11:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.120+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1ade30 con 0x7f640c110280 2024-05-27T07:11:04.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.130+0000 7f6402a00700 1 -- 172.21.15.16:0/3331658740 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f640401b070 con 0x7f640c110280 2024-05-27T07:11:04.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.133+0000 7f6402a00700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 0x7f63f00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.133+0000 7f6402a00700 1 -- 172.21.15.16:0/3331658740 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f64040e1fd0 con 0x7f640c110280 2024-05-27T07:11:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.133+0000 7f6409e00700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 0x7f63f00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.135+0000 7f6409e00700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 0x7f63f00b93c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009d20 tx=0x7f63fc009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.153+0000 7f641fd20b80 1 Processor -- start 2024-05-27T07:11:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.154+0000 7f641fd20b80 1 -- start start 2024-05-27T07:11:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff83002b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82e7be40 0x55ff83003090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff8300f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff82f28e40 con 0x55ff82e7be40 2024-05-27T07:11:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff82f189e0 con 0x55ff82f679f0 2024-05-27T07:11:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.155+0000 7f641fd20b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ff82f1ca20 con 0x55ff83003690 2024-05-27T07:11:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.156+0000 7f6409e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff83002b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.156+0000 7f6409e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff83002b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58752/0 (socket says 172.21.15.16:58752) 2024-05-27T07:11:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.156+0000 7f6409e00700 1 -- 172.21.15.16:0/4111574957 learned_addr learned my addr 172.21.15.16:0/4111574957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.156+0000 7f6409400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82e7be40 0x55ff83003090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.157+0000 7f640a800700 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff8300f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.158+0000 7f640a800700 1 -- 172.21.15.16:0/4111574957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 msgr2=0x55ff83002b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.158+0000 7f640a800700 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff83002b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.158+0000 7f640a800700 1 -- 172.21.15.16:0/4111574957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82e7be40 msgr2=0x55ff83003090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.158+0000 7f640a800700 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82e7be40 0x55ff83003090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.158+0000 7f640a800700 1 -- 172.21.15.16:0/4111574957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff82ee1420 con 0x55ff83003690 2024-05-27T07:11:04.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.160+0000 7f640a800700 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff8300f920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64040df340 tx=0x7f64040a8a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8ad9542b5c85b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.162+0000 7f63f7400700 1 -- 172.21.15.16:0/4111574957 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404009a40 con 0x55ff83003690 2024-05-27T07:11:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.162+0000 7f63f7400700 1 -- 172.21.15.16:0/4111574957 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64040cb040 con 0x55ff83003690 2024-05-27T07:11:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.162+0000 7f63f7400700 1 -- 172.21.15.16:0/4111574957 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040b1ae0 con 0x55ff83003690 2024-05-27T07:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.167+0000 7f641fd20b80 1 -- 172.21.15.16:0/4111574957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 msgr2=0x55ff8300f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.167+0000 7f641fd20b80 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff8300f920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64040df340 tx=0x7f64040a8a60 comp rx=0 tx=0).stop 2024-05-27T07:11:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.168+0000 7f641fd20b80 1 -- 172.21.15.16:0/4111574957 shutdown_connections 2024-05-27T07:11:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.168+0000 7f641fd20b80 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff8300f920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.168+0000 7f641fd20b80 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff83002b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.168+0000 7f641fd20b80 1 --2- 172.21.15.16:0/4111574957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82e7be40 0x55ff83003090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.169+0000 7f641fd20b80 1 -- 172.21.15.16:0/4111574957 >> 172.21.15.16:0/4111574957 conn(0x55ff82f15390 msgr2=0x55ff82ff6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.171+0000 7f641fd20b80 1 -- 172.21.15.16:0/4111574957 shutdown_connections 2024-05-27T07:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.171+0000 7f641fd20b80 1 -- 172.21.15.16:0/4111574957 wait complete. 2024-05-27T07:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.172+0000 7f641fd20b80 1 -- start start 2024-05-27T07:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.172+0000 7f641fd20b80 1 Processor -- start 2024-05-27T07:11:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.173+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.173+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.173+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff8300da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.176+0000 7f6409400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.176+0000 7f6409400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58760/0 (socket says 172.21.15.16:58760) 2024-05-27T07:11:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.177+0000 7f640a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff8300da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.177+0000 7f640a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff8300da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45778/0 (socket says 172.21.15.16:45778) 2024-05-27T07:11:04.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f6409e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f6409e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59370/0 (socket says 172.21.15.16:59370) 2024-05-27T07:11:04.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff82e1efa0 con 0x55ff83003690 2024-05-27T07:11:04.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff82eb9760 con 0x55ff82f679f0 2024-05-27T07:11:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ff830103a0 con 0x55ff82e7be40 2024-05-27T07:11:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f641fd20b80 10 client.0 _ll_register_callbacks cb 0x55ff82ec2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.178+0000 7f6409400700 1 -- 172.21.15.16:0/98249582 learned_addr learned my addr 172.21.15.16:0/98249582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.181+0000 7f6409e00700 1 -- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 msgr2=0x55ff8300d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.181+0000 7f6409e00700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.181+0000 7f6409e00700 1 -- 172.21.15.16:0/98249582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 msgr2=0x55ff8300da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.181+0000 7f6409e00700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff8300da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.181+0000 7f6409e00700 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff82e688d0 con 0x55ff82e7be40 2024-05-27T07:11:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.182+0000 7f6409400700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.183+0000 7f6409e00700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00ec60 tx=0x7f63fc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ed4bbb0c5b924 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.183+0000 7f63ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.183+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc02b060 con 0x55ff82e7be40 2024-05-27T07:11:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.183+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64040cb040 con 0x55ff82e7be40 2024-05-27T07:11:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.184+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f63fc01dae0 con 0x55ff82e7be40 2024-05-27T07:11:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.185+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63fc032050 con 0x55ff82e7be40 2024-05-27T07:11:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.185+0000 7f641fd20b80 10 client.26215 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.185+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ff82ee9060 con 0x55ff82e7be40 2024-05-27T07:11:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.185+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64040cb040 con 0x55ff82e7be40 2024-05-27T07:11:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.189+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f63fc01c400 con 0x55ff82e7be40 2024-05-27T07:11:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.189+0000 7f641fd20b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.189+0000 7f641fd20b80 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:04.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.189+0000 7f641fd20b80 20 client.26215 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.189+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff8301d350 0x55ff8301f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.190+0000 7f641fd20b80 4 client.26215 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:11:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.190+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff82f2ad90 con 0x55ff8301d350 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.191+0000 7f6409400700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff8301d350 0x55ff8301f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.192+0000 7f6409400700 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff8301d350 0x55ff8301f790 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a2e60c04d16f2 server_cookie=ac2057e84e57decf in_seq=0 out_seq=0 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.192+0000 7f63ebe00700 10 client.26215 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.193+0000 7f63ebe00700 1 -- 172.21.15.16:0/98249582 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ff82f2ad90 con 0x55ff8301d350 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.193+0000 7f63ebe00700 10 client.26215 handle_command_reply: tid=0 2024-05-27T07:11:04.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.193+0000 7f641fd20b80 1 client.26215 shutdown 2024-05-27T07:11:04.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.194+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 msgr2=0x55ff8300d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.194+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00ec60 tx=0x7f63fc0047c0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 shutdown_connections 2024-05-27T07:11:04.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff8301d350 0x55ff8301f790 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff8300da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82e7be40 0x55ff8300d000 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 --2- 172.21.15.16:0/98249582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82f679f0 0x55ff8300d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f63ebe00700 0 client.26215 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:04.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 >> 172.21.15.16:0/98249582 conn(0x55ff82f15390 msgr2=0x55ff83002d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 shutdown_connections 2024-05-27T07:11:04.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.196+0000 7f641fd20b80 1 -- 172.21.15.16:0/98249582 wait complete. 2024-05-27T07:11:04.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.197+0000 7f641fd20b80 20 client.26215 trim_cache size 0 max 16384 2024-05-27T07:11:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.238+0000 7f641fd20b80 1 Processor -- start 2024-05-27T07:11:04.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.238+0000 7f641fd20b80 1 -- start start 2024-05-27T07:11:04.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83008840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff83019440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b6c50 con 0x55ff83003690 2024-05-27T07:11:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b8790 con 0x55ff82e7be40 2024-05-27T07:11:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.239+0000 7f641fd20b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b7320 con 0x55ff82f679f0 2024-05-27T07:11:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.241+0000 7f6409400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.241+0000 7f640a800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff83019440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.241+0000 7f6409400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59376/0 (socket says 172.21.15.16:59376) 2024-05-27T07:11:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.241+0000 7f6409400700 1 -- 172.21.15.16:0/3187215406 learned_addr learned my addr 172.21.15.16:0/3187215406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.241+0000 7f6409e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83008840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.243+0000 7f6409e00700 1 -- 172.21.15.16:0/3187215406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 msgr2=0x55ff8300bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.243+0000 7f6409e00700 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.243+0000 7f6409e00700 1 -- 172.21.15.16:0/3187215406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 msgr2=0x55ff83019440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.243+0000 7f6409e00700 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff83019440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.243+0000 7f6409e00700 1 -- 172.21.15.16:0/3187215406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff8300ddf0 con 0x55ff82e7be40 2024-05-27T07:11:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.244+0000 7f6409400700 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.245+0000 7f6409e00700 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83008840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f63fc01c7d0 tx=0x7f63fc01d7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b196428939973b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.245+0000 7f63f6a00700 1 -- 172.21.15.16:0/3187215406 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc03b070 con 0x55ff82e7be40 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.245+0000 7f63f6a00700 1 -- 172.21.15.16:0/3187215406 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63fc01c400 con 0x55ff82e7be40 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.245+0000 7f63f6a00700 1 -- 172.21.15.16:0/3187215406 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc02b040 con 0x55ff82e7be40 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.247+0000 7f641fd20b80 1 -- 172.21.15.16:0/3187215406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 msgr2=0x55ff83008840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.247+0000 7f641fd20b80 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83008840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f63fc01c7d0 tx=0x7f63fc01d7a0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.248+0000 7f641fd20b80 1 -- 172.21.15.16:0/3187215406 shutdown_connections 2024-05-27T07:11:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.248+0000 7f641fd20b80 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff83003690 0x55ff83019440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.249+0000 7f641fd20b80 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83008840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.249+0000 7f641fd20b80 1 --2- 172.21.15.16:0/3187215406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff82f679f0 0x55ff8300bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.250+0000 7f641fd20b80 1 -- 172.21.15.16:0/3187215406 >> 172.21.15.16:0/3187215406 conn(0x55ff82f15390 msgr2=0x55ff830046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.250+0000 7f641fd20b80 1 -- 172.21.15.16:0/3187215406 shutdown_connections 2024-05-27T07:11:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.250+0000 7f641fd20b80 1 -- 172.21.15.16:0/3187215406 wait complete. 2024-05-27T07:11:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.251+0000 7f641fd20b80 1 -- start start 2024-05-27T07:11:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.251+0000 7f641fd20b80 1 Processor -- start 2024-05-27T07:11:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.253+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.254+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.255+0000 7f641fd20b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f641fd20b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b80c0 con 0x55ff82f679f0 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f641fd20b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b79f0 con 0x55ff82e7be40 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f641fd20b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b4c10 con 0x55ff83003690 2024-05-27T07:11:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f641fd20b80 10 client.0 _ll_register_callbacks cb 0x55ff82ec2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f6409400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.257+0000 7f640a800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.258+0000 7f640a800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:59392/0 (socket says 172.21.15.16:59392) 2024-05-27T07:11:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.256+0000 7f6409e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.258+0000 7f6409e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:58788/0 (socket says 172.21.15.16:58788) 2024-05-27T07:11:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.259+0000 7f6409e00700 1 -- 172.21.15.16:0/1001514192 learned_addr learned my addr 172.21.15.16:0/1001514192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.258+0000 7f6409400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45788/0 (socket says 172.21.15.16:45788) 2024-05-27T07:11:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.260+0000 7f6409400700 1 -- 172.21.15.16:0/1001514192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 msgr2=0x55ff83007ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.260+0000 7f6409400700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.260+0000 7f6409400700 1 -- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 msgr2=0x55ff83007010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.260+0000 7f6409400700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.260+0000 7f6409400700 1 -- 172.21.15.16:0/1001514192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff83014890 con 0x55ff82f679f0 2024-05-27T07:11:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.261+0000 7f640a800700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.261+0000 7f6409400700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f63f80098e0 tx=0x7f63f8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd294878ce10654 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.261+0000 7f6409e00700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.261+0000 7f63ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:11:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.262+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8003df0 con 0x55ff82f679f0 2024-05-27T07:11:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.263+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9509+0+0 (secure 0 0 0) 0x7f63f800b6b0 con 0x55ff82f679f0 2024-05-27T07:11:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.263+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc01c400 con 0x55ff82f679f0 2024-05-27T07:11:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.264+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63f800b8b0 con 0x55ff82f679f0 2024-05-27T07:11:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.264+0000 7f641fd20b80 10 client.16398 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.264+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ff83015b50 con 0x55ff82f679f0 2024-05-27T07:11:04.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63fc01c400 con 0x55ff82f679f0 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f63f802f060 con 0x55ff82f679f0 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 20 client.16398 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff83087e20 0x55ff8308a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 4 client.16398 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.266+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff82f24570 con 0x55ff83087e20 2024-05-27T07:11:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.269+0000 7f6409e00700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff83087e20 0x55ff8308a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.270+0000 7f6409e00700 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff83087e20 0x55ff8308a240 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3319051892d7ba4b server_cookie=e72663ad761e57e4 in_seq=0 out_seq=0 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.271+0000 7f63ebe00700 10 client.16398 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.271+0000 7f63ebe00700 1 -- 172.21.15.16:0/1001514192 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff82f24570 con 0x55ff83087e20 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.271+0000 7f63ebe00700 10 client.16398 handle_command_reply: tid=0 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.271+0000 7f641fd20b80 1 client.16398 shutdown 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.272+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 msgr2=0x55ff83007550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.272+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f63f80098e0 tx=0x7f63f8009700 comp rx=0 tx=0).stop 2024-05-27T07:11:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.274+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 shutdown_connections 2024-05-27T07:11:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.274+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ff83087e20 0x55ff8308a240 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.274+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ff83003690 0x55ff83007ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff82e7be40 0x55ff83007010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 1 --2- 172.21.15.16:0/1001514192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff82f679f0 0x55ff83007550 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 >> 172.21.15.16:0/1001514192 conn(0x55ff82f15390 msgr2=0x55ff8300e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f63ebe00700 0 client.16398 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 shutdown_connections 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 1 -- 172.21.15.16:0/1001514192 wait complete. 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.275+0000 7f641fd20b80 20 client.16398 trim_cache size 0 max 16384 2024-05-27T07:11:04.267 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.277+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 msgr2=0x7f63f00b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.277+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 0x7f63f00b93c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009d20 tx=0x7f63fc009260 comp rx=0 tx=0).stop 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.277+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 msgr2=0x7f640c1a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.277+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f64040049d0 tx=0x7f640400a320 comp rx=0 tx=0).stop 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 shutdown_connections 2024-05-27T07:11:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f63f00b6eb0 0x7f63f00b93c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c110280 0x7f640c1a5b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f640c10b000 0x7f640c1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 --2- 172.21.15.16:0/3331658740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10f930 0x7f640c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 >> 172.21.15.16:0/3331658740 conn(0x7f640c06f4c0 msgr2=0x7f640c10da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 shutdown_connections 2024-05-27T07:11:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:04.278+0000 7f6411200700 1 -- 172.21.15.16:0/3331658740 wait complete. 2024-05-27T07:11:04.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:03 smithi164 ceph-mon[35327]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 27 MiB/s wr, 564 op/s 2024-05-27T07:11:04.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:03 smithi069 ceph-mon[35276]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 27 MiB/s wr, 564 op/s 2024-05-27T07:11:06.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:05 smithi164 ceph-mon[35327]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 515 op/s 2024-05-27T07:11:06.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:05 smithi016 ceph-mon[31415]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 515 op/s 2024-05-27T07:11:06.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:05 smithi069 ceph-mon[35276]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 515 op/s 2024-05-27T07:11:08.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:07 smithi164 ceph-mon[35327]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 24 MiB/s wr, 524 op/s 2024-05-27T07:11:08.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:07 smithi016 ceph-mon[31415]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 24 MiB/s wr, 524 op/s 2024-05-27T07:11:08.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:07 smithi069 ceph-mon[35276]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 24 MiB/s wr, 524 op/s 2024-05-27T07:11:09.114 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:11:09.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:11:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:08 smithi016 ceph-mon[31415]: pgmap v318: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 497 op/s 2024-05-27T07:11:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:08 smithi016 ceph-mon[31415]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:11:09.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:08 smithi164 ceph-mon[35327]: pgmap v318: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 497 op/s 2024-05-27T07:11:09.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:08 smithi164 ceph-mon[35327]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:11:09.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:08 smithi069 ceph-mon[35276]: pgmap v318: 129 pgs: 129 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 497 op/s 2024-05-27T07:11:09.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:08 smithi069 ceph-mon[35276]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:11:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:11:11.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:11 smithi016 ceph-mon[31415]: pgmap v320: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-05-27T07:11:11.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:11 smithi069 ceph-mon[35276]: pgmap v320: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-05-27T07:11:11.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:11 smithi164 ceph-mon[35327]: pgmap v320: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-05-27T07:11:13.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:13 smithi016 ceph-mon[31415]: pgmap v321: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 482 op/s 2024-05-27T07:11:13.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:13 smithi069 ceph-mon[35276]: pgmap v321: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 482 op/s 2024-05-27T07:11:13.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:13 smithi164 ceph-mon[35327]: pgmap v321: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 20 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 482 op/s 2024-05-27T07:11:15.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:15 smithi016 ceph-mon[31415]: pgmap v322: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:11:15.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:15 smithi069 ceph-mon[35276]: pgmap v322: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:11:15.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:15 smithi164 ceph-mon[35327]: pgmap v322: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 20 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:11:17.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:17 smithi016 ceph-mon[31415]: pgmap v323: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 389 op/s 2024-05-27T07:11:17.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:17 smithi069 ceph-mon[35276]: pgmap v323: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 389 op/s 2024-05-27T07:11:17.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:17 smithi164 ceph-mon[35327]: pgmap v323: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 389 op/s 2024-05-27T07:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.790+0000 7ff1f3e00700 1 -- 172.21.15.16:0/3287868244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec1030b0 msgr2=0x7ff1ec1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.790+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/3287868244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1034c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009990 tx=0x7ff1e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 -- 172.21.15.16:0/3287868244 shutdown_connections 2024-05-27T07:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/3287868244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec109280 0x7ff1ec117460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/3287868244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1034c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/3287868244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec103a00 0x7ff1ec114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 -- 172.21.15.16:0/3287868244 >> 172.21.15.16:0/3287868244 conn(0x7ff1ec0fedb0 msgr2=0x7ff1ec1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 -- 172.21.15.16:0/3287868244 shutdown_connections 2024-05-27T07:11:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.792+0000 7ff1f3e00700 1 -- 172.21.15.16:0/3287868244 wait complete. 2024-05-27T07:11:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.794+0000 7ff1f3e00700 1 Processor -- start 2024-05-27T07:11:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.794+0000 7ff1f3e00700 1 -- start start 2024-05-27T07:11:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.794+0000 7ff1f3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.794+0000 7ff1f3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 0x7ff1ec1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.795+0000 7ff1f3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 0x7ff1ec1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.796+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.796+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49344/0 (socket says 172.21.15.16:49344) 2024-05-27T07:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.796+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 0x7ff1ec1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.796+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 0x7ff1ec1a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:47998/0 (socket says 172.21.15.16:47998) 2024-05-27T07:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.797+0000 7ff1f3e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec11a9e0 con 0x7ff1ec1030b0 2024-05-27T07:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.797+0000 7ff1f3e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec11a860 con 0x7ff1ec103a00 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.797+0000 7ff1f3e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec11ab60 con 0x7ff1ec109280 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.797+0000 7ff1f1400700 1 -- 172.21.15.16:0/1616524689 learned_addr learned my addr 172.21.15.16:0/1616524689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.797+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 0x7ff1ec1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 -- 172.21.15.16:0/1616524689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 msgr2=0x7ff1ec1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 0x7ff1ec1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 -- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 msgr2=0x7ff1ec1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 -- 172.21.15.16:0/1616524689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1a62c0 con 0x7ff1ec109280 2024-05-27T07:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.798+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 0x7ff1ec1a5c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800e970 tx=0x7ff1d800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.799+0000 7ff1ea000700 1 -- 172.21.15.16:0/1616524689 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800cbd0 con 0x7ff1ec109280 2024-05-27T07:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.799+0000 7ff1ea000700 1 -- 172.21.15.16:0/1616524689 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1d80044b0 con 0x7ff1ec109280 2024-05-27T07:11:19.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.799+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1a65b0 con 0x7ff1ec109280 2024-05-27T07:11:19.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.799+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1aded0 con 0x7ff1ec109280 2024-05-27T07:11:19.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.799+0000 7ff1ea000700 1 -- 172.21.15.16:0/1616524689 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8005020 con 0x7ff1ec109280 2024-05-27T07:11:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.811+0000 7ff1ea000700 1 -- 172.21.15.16:0/1616524689 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff1d8004750 con 0x7ff1ec109280 2024-05-27T07:11:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.813+0000 7ff1ea000700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 0x7ff1d40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.814+0000 7ff1ea000700 1 -- 172.21.15.16:0/1616524689 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7ff1d80e2820 con 0x7ff1ec109280 2024-05-27T07:11:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.814+0000 7ff1f1400700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 0x7ff1d40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.817+0000 7ff1f1400700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 0x7ff1d40b5000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b2e0 tx=0x7ff1e4005fb0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.843+0000 7ff20291cb80 1 Processor -- start 2024-05-27T07:11:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.843+0000 7ff20291cb80 1 -- start start 2024-05-27T07:11:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b43939870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b439338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b43861ec0 con 0x560b437b4ec0 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b43851a60 con 0x560b43933e30 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff20291cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560b43855aa0 con 0x560b43939db0 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.844+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b439338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.845+0000 7ff1f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.845+0000 7ff1f1e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.845+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2115283406 learned_addr learned my addr 172.21.15.16:0/2115283406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.846+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2115283406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 msgr2=0x560b439338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.846+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b439338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.846+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2115283406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 msgr2=0x560b43939870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.846+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b43939870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.846+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2115283406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4381a4a0 con 0x560b43933e30 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.847+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80a9360 tx=0x7ff1d80a9e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbcfe86b2e0c0602 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.847+0000 7ff1dea00700 1 -- 172.21.15.16:0/2115283406 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d801c040 con 0x560b43933e30 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.847+0000 7ff1dea00700 1 -- 172.21.15.16:0/2115283406 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1d80b24c0 con 0x560b43933e30 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.847+0000 7ff1dea00700 1 -- 172.21.15.16:0/2115283406 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80b2270 con 0x560b43933e30 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.849+0000 7ff20291cb80 1 -- 172.21.15.16:0/2115283406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 msgr2=0x560b43934240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.849+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80a9360 tx=0x7ff1d80a9e30 comp rx=0 tx=0).stop 2024-05-27T07:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.849+0000 7ff20291cb80 1 -- 172.21.15.16:0/2115283406 shutdown_connections 2024-05-27T07:11:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.849+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b43934240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.849+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b43939870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.850+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2115283406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b439338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.850+0000 7ff20291cb80 1 -- 172.21.15.16:0/2115283406 >> 172.21.15.16:0/2115283406 conn(0x560b43815040 msgr2=0x560b4392e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.852+0000 7ff20291cb80 1 -- 172.21.15.16:0/2115283406 shutdown_connections 2024-05-27T07:11:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.852+0000 7ff20291cb80 1 -- 172.21.15.16:0/2115283406 wait complete. 2024-05-27T07:11:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.853+0000 7ff20291cb80 1 -- start start 2024-05-27T07:11:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.853+0000 7ff20291cb80 1 Processor -- start 2024-05-27T07:11:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.855+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b4393d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.855+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b4393dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.856+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b4393d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.856+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b4393d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49370/0 (socket says 172.21.15.16:49370) 2024-05-27T07:11:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.856+0000 7ff1f1400700 1 -- 172.21.15.16:0/2031439594 learned_addr learned my addr 172.21.15.16:0/2031439594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.856+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b4393dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.856+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b4393dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.857+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b43758020 con 0x560b437b4ec0 2024-05-27T07:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.857+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b437f27e0 con 0x560b43933e30 2024-05-27T07:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.857+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560b43940880 con 0x560b43939db0 2024-05-27T07:11:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.857+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b4393dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.857+0000 7ff20291cb80 10 client.0 _ll_register_callbacks cb 0x560b437fbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 msgr2=0x560b4393dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b4393dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2031439594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 msgr2=0x560b4393d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b4393d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b437a1950 con 0x560b43939db0 2024-05-27T07:11:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.859+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b4393dff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560b4393a3a0 tx=0x7ff1d80a9520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26c946dc35d1a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.860+0000 7ff1d3400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.863+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80c8060 con 0x560b43939db0 2024-05-27T07:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.863+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7ff1d80e2360 con 0x560b43939db0 2024-05-27T07:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.863+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d80b24c0 con 0x560b43939db0 2024-05-27T07:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.864+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1d801c040 con 0x560b43939db0 2024-05-27T07:11:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.865+0000 7ff20291cb80 10 client.26230 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.865+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b438220e0 con 0x560b43939db0 2024-05-27T07:11:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.865+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1d80b24c0 con 0x560b43939db0 2024-05-27T07:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff1d80a4fd0 con 0x560b43939db0 2024-05-27T07:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 10 client.26230 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 10 client.26230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 20 client.26230 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b4394d660 0x560b4394fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 4 client.26230 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.867+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b43863e10 con 0x560b4394d660 2024-05-27T07:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.868+0000 7ff1f1400700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b4394d660 0x560b4394fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.869+0000 7ff1f1400700 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b4394d660 0x560b4394fa80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd6e01bc0cbdcc6 server_cookie=ef2cc2ec8ab32555 in_seq=0 out_seq=0 2024-05-27T07:11:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.869+0000 7ff1d3400700 10 client.26230 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.870+0000 7ff1d3400700 1 -- 172.21.15.16:0/2031439594 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560b43863e10 con 0x560b4394d660 2024-05-27T07:11:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.870+0000 7ff1d3400700 10 client.26230 handle_command_reply: tid=0 2024-05-27T07:11:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.870+0000 7ff20291cb80 1 client.26230 shutdown 2024-05-27T07:11:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.871+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 msgr2=0x560b4393dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.871+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b4393dff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560b4393a3a0 tx=0x7ff1d80a9520 comp rx=0 tx=0).stop 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 shutdown_connections 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b4394d660 0x560b4394fa80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff1d3400700 0 client.26230 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43939db0 0x560b4393dff0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b437b4ec0 0x560b4393d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2031439594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43933e30 0x560b4393dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.872+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 >> 172.21.15.16:0/2031439594 conn(0x560b43815040 msgr2=0x560b4392e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.873+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 shutdown_connections 2024-05-27T07:11:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.874+0000 7ff20291cb80 1 -- 172.21.15.16:0/2031439594 wait complete. 2024-05-27T07:11:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.874+0000 7ff20291cb80 20 client.26230 trim_cache size 0 max 16384 2024-05-27T07:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.919+0000 7ff20291cb80 1 Processor -- start 2024-05-27T07:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.919+0000 7ff20291cb80 1 -- start start 2024-05-27T07:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b437b4ec0 0x560b439338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 0x560b43949ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc00efe0 con 0x560b43939db0 2024-05-27T07:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc010b20 con 0x560b437b4ec0 2024-05-27T07:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff20291cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc00f6b0 con 0x560b43933e30 2024-05-27T07:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.920+0000 7ff1f1e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 0x560b43949ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.921+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.921+0000 7ff1f0a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43854/0 (socket says 172.21.15.16:43854) 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.921+0000 7ff1f0a00700 1 -- 172.21.15.16:0/2932695968 learned_addr learned my addr 172.21.15.16:0/2932695968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.921+0000 7ff1f1400700 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b437b4ec0 0x560b439338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.922+0000 7ff1f0a00700 1 -- 172.21.15.16:0/2932695968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b437b4ec0 msgr2=0x560b439338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.922+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b437b4ec0 0x560b439338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.922+0000 7ff1f0a00700 1 -- 172.21.15.16:0/2932695968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 msgr2=0x560b43949ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.922+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 0x560b43949ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.922+0000 7ff1f0a00700 1 -- 172.21.15.16:0/2932695968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b439402d0 con 0x560b43933e30 2024-05-27T07:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.923+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 0x560b43949ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.923+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009f60 tx=0x7ff1e0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f496277f89b369d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.924+0000 7ff1de000700 1 -- 172.21.15.16:0/2932695968 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e001a020 con 0x560b43933e30 2024-05-27T07:11:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.924+0000 7ff1de000700 1 -- 172.21.15.16:0/2932695968 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e000ae20 con 0x560b43933e30 2024-05-27T07:11:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.924+0000 7ff1de000700 1 -- 172.21.15.16:0/2932695968 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e001a020 con 0x560b43933e30 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.925+0000 7ff20291cb80 1 -- 172.21.15.16:0/2932695968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 msgr2=0x560b4393fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.925+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009f60 tx=0x7ff1e0009f90 comp rx=0 tx=0).stop 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.926+0000 7ff20291cb80 1 -- 172.21.15.16:0/2932695968 shutdown_connections 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.926+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43939db0 0x560b43949ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.926+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b437b4ec0 0x560b439338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.926+0000 7ff20291cb80 1 --2- 172.21.15.16:0/2932695968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b43933e30 0x560b4393fa10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.926+0000 7ff20291cb80 1 -- 172.21.15.16:0/2932695968 >> 172.21.15.16:0/2932695968 conn(0x560b43815040 msgr2=0x560b4393dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.927+0000 7ff20291cb80 1 -- 172.21.15.16:0/2932695968 shutdown_connections 2024-05-27T07:11:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.927+0000 7ff20291cb80 1 -- 172.21.15.16:0/2932695968 wait complete. 2024-05-27T07:11:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.928+0000 7ff20291cb80 1 -- start start 2024-05-27T07:11:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.928+0000 7ff20291cb80 1 Processor -- start 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.929+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.929+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 0x560b43951200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.929+0000 7ff20291cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43939db0 0x560b43951740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.929+0000 7ff20291cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc010450 con 0x560b43933e30 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.929+0000 7ff20291cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc00fd80 con 0x560b43939db0 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.930+0000 7ff20291cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc00cfa0 con 0x560b437b4ec0 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.930+0000 7ff20291cb80 10 client.0 _ll_register_callbacks cb 0x560b437fbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.931+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.931+0000 7ff1f1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:43864/0 (socket says 172.21.15.16:43864) 2024-05-27T07:11:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.931+0000 7ff1f1400700 1 -- 172.21.15.16:0/1098389788 learned_addr learned my addr 172.21.15.16:0/1098389788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.932+0000 7ff1f1e00700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43939db0 0x560b43951740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.933+0000 7ff1f1400700 1 -- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43939db0 msgr2=0x560b43951740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.933+0000 7ff1f1400700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43939db0 0x560b43951740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.933+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 0x560b43951200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.934+0000 7ff1f1400700 1 -- 172.21.15.16:0/1098389788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 msgr2=0x560b43951200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.934+0000 7ff1f1400700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 0x560b43951200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.934+0000 7ff1f1400700 1 -- 172.21.15.16:0/1098389788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b43951c80 con 0x560b437b4ec0 2024-05-27T07:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.935+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 0x560b43951200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:11:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.936+0000 7ff1f1400700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b650 tx=0x7ff1e4003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f264b11d17c0d427 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.936+0000 7ff1d3400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.937+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400bb50 con 0x560b437b4ec0 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.937+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7ff1e400bcf0 con 0x560b437b4ec0 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.937+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e000ae20 con 0x560b437b4ec0 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.938+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1e403f050 con 0x560b437b4ec0 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.938+0000 7ff20291cb80 10 client.26236 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.938+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b439c9b60 con 0x560b437b4ec0 2024-05-27T07:11:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.938+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e000ae20 con 0x560b437b4ec0 2024-05-27T07:11:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e400f460 con 0x560b437b4ec0 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 10 client.26236 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 20 client.26236 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b439ca270 0x560b439cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 4 client.26236 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b4385d5f0 con 0x560b439ca270 2024-05-27T07:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.939+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b439ca270 0x560b439cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.941+0000 7ff1f0a00700 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b439ca270 0x560b439cc690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247d4140a49986df server_cookie=1fa29e505c0556df in_seq=0 out_seq=0 2024-05-27T07:11:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.941+0000 7ff1d3400700 10 client.26236 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.944+0000 7ff1d3400700 1 -- 172.21.15.16:0/1098389788 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b4385d5f0 con 0x560b439ca270 2024-05-27T07:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.944+0000 7ff1d3400700 10 client.26236 handle_command_reply: tid=0 2024-05-27T07:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.944+0000 7ff20291cb80 1 client.26236 shutdown 2024-05-27T07:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 msgr2=0x560b43952e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b650 tx=0x7ff1e4003e40 comp rx=0 tx=0).stop 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 shutdown_connections 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x560b439ca270 0x560b439cc690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b43939db0 0x560b43951740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b437b4ec0 0x560b43952e70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 --2- 172.21.15.16:0/1098389788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b43933e30 0x560b43951200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff1d3400700 0 client.26236 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 >> 172.21.15.16:0/1098389788 conn(0x560b43815040 msgr2=0x560b4393dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 shutdown_connections 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.945+0000 7ff20291cb80 1 -- 172.21.15.16:0/1098389788 wait complete. 2024-05-27T07:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.946+0000 7ff20291cb80 20 client.26236 trim_cache size 0 max 16384 2024-05-27T07:11:19.938 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:11:19.938 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:11:19.938 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7f6f4242-1d17-471c-87dd-a8fad945820b", 2024-05-27T07:11:19.938 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:11:19.938 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 msgr2=0x7ff1d40b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 0x7ff1d40b5000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b2e0 tx=0x7ff1e4005fb0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 msgr2=0x7ff1ec1a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 0x7ff1ec1a5c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800e970 tx=0x7ff1d800ed30 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 shutdown_connections 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff1d40b2af0 0x7ff1d40b5000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1ec109280 0x7ff1ec1a5c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec1030b0 0x7ff1ec1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 --2- 172.21.15.16:0/1616524689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1ec103a00 0x7ff1ec1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.948+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 >> 172.21.15.16:0/1616524689 conn(0x7ff1ec0fedb0 msgr2=0x7ff1ec116440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.949+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 shutdown_connections 2024-05-27T07:11:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:19.949+0000 7ff1f3e00700 1 -- 172.21.15.16:0/1616524689 wait complete. 2024-05-27T07:11:20.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:19 smithi164 ceph-mon[35327]: pgmap v324: 129 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 94 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-05-27T07:11:20.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:19 smithi016 ceph-mon[31415]: pgmap v324: 129 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 94 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-05-27T07:11:20.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:19 smithi069 ceph-mon[35276]: pgmap v324: 129 pgs: 10 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 94 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-05-27T07:11:21.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:11:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:21 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:11:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:21 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:21 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:11:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:21 smithi164 ceph-mon[35327]: pgmap v325: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 338 op/s 2024-05-27T07:11:22.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:21 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:21 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:11:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:21 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:21 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:11:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:21 smithi016 ceph-mon[31415]: pgmap v325: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 338 op/s 2024-05-27T07:11:22.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:21 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:22.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:21 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:11:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:21 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:21 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:11:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:21 smithi069 ceph-mon[35276]: pgmap v325: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 338 op/s 2024-05-27T07:11:22.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:21 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:11:24.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:23 smithi164 ceph-mon[35327]: pgmap v326: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 391 op/s 2024-05-27T07:11:24.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:23 smithi016 ceph-mon[31415]: pgmap v326: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 391 op/s 2024-05-27T07:11:24.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:23 smithi069 ceph-mon[35276]: pgmap v326: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 391 op/s 2024-05-27T07:11:25.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:24 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:11:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:24 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:11:25.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:24 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:11:25.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:25 smithi016 ceph-mon[31415]: pgmap v327: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:11:26.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:25 smithi164 ceph-mon[35327]: pgmap v327: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:11:26.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:25 smithi069 ceph-mon[35276]: pgmap v327: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:11:28.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:27 smithi164 ceph-mon[35327]: pgmap v328: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 386 op/s 2024-05-27T07:11:28.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:27 smithi016 ceph-mon[31415]: pgmap v328: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 386 op/s 2024-05-27T07:11:28.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:27 smithi069 ceph-mon[35276]: pgmap v328: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 386 op/s 2024-05-27T07:11:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:29 smithi164 ceph-mon[35327]: pgmap v329: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 369 op/s 2024-05-27T07:11:30.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:29 smithi016 ceph-mon[31415]: pgmap v329: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 369 op/s 2024-05-27T07:11:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:29 smithi069 ceph-mon[35276]: pgmap v329: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 369 op/s 2024-05-27T07:11:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.670+0000 7f1a07400700 1 -- 172.21.15.16:0/3670397969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 msgr2=0x7f1a00109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.670+0000 7f1a07400700 1 --2- 172.21.15.16:0/3670397969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a00109320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19ec013ef0 tx=0x7f19ec0097f0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.673+0000 7f1a07400700 1 -- 172.21.15.16:0/3670397969 shutdown_connections 2024-05-27T07:11:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.673+0000 7f1a07400700 1 --2- 172.21.15.16:0/3670397969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a00109860 0x7f1a001111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.673+0000 7f1a07400700 1 --2- 172.21.15.16:0/3670397969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a001045e0 0x7f1a001049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.673+0000 7f1a07400700 1 --2- 172.21.15.16:0/3670397969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a00109320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.673+0000 7f1a07400700 1 -- 172.21.15.16:0/3670397969 >> 172.21.15.16:0/3670397969 conn(0x7f1a00100030 msgr2=0x7f1a00102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.675+0000 7f1a07400700 1 -- 172.21.15.16:0/3670397969 shutdown_connections 2024-05-27T07:11:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.676+0000 7f1a07400700 1 -- 172.21.15.16:0/3670397969 wait complete. 2024-05-27T07:11:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.677+0000 7f1a07400700 1 Processor -- start 2024-05-27T07:11:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.681+0000 7f1a07400700 1 -- start start 2024-05-27T07:11:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.682+0000 7f1a07400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 0x7f1a0006f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.682+0000 7f1a07400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a0006f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.682+0000 7f1a07400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a07400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0006d560 con 0x7f1a001045e0 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a05400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a07400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0006d3e0 con 0x7f1a00109860 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a07400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0006d6e0 con 0x7f1a00108f10 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a05400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:46244/0 (socket says 172.21.15.16:46244) 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a05400700 1 -- 172.21.15.16:0/1323195131 learned_addr learned my addr 172.21.15.16:0/1323195131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f1a04a00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 0x7f1a0006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.684+0000 7f1a04a00700 1 -- 172.21.15.16:0/1323195131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 msgr2=0x7f1a0006f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.683+0000 7f19ffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a0006f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.685+0000 7f1a04a00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a0006f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.685+0000 7f1a04a00700 1 -- 172.21.15.16:0/1323195131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 msgr2=0x7f1a000749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.685+0000 7f1a04a00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.685+0000 7f1a04a00700 1 -- 172.21.15.16:0/1323195131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a000751d0 con 0x7f1a001045e0 2024-05-27T07:11:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.686+0000 7f1a05400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.686+0000 7f19ffe00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a0006f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:11:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.687+0000 7f1a04a00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 0x7f1a0006f240 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.688+0000 7f19fd600700 1 -- 172.21.15.16:0/1323195131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0017070 con 0x7f1a001045e0 2024-05-27T07:11:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.688+0000 7f19fd600700 1 -- 172.21.15.16:0/1323195131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f0009dc0 con 0x7f1a001045e0 2024-05-27T07:11:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.688+0000 7f19fd600700 1 -- 172.21.15.16:0/1323195131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0010cd0 con 0x7f1a001045e0 2024-05-27T07:11:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.689+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a000754c0 con 0x7f1a001045e0 2024-05-27T07:11:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.689+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001adb50 con 0x7f1a001045e0 2024-05-27T07:11:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.699+0000 7f19fd600700 1 -- 172.21.15.16:0/1323195131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f19f001a070 con 0x7f1a001045e0 2024-05-27T07:11:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.703+0000 7f19fd600700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 0x7f19e40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.703+0000 7f19fd600700 1 -- 172.21.15.16:0/1323195131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7f19f00e2420 con 0x7f1a001045e0 2024-05-27T07:11:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.704+0000 7f19ffe00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 0x7f19e40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.706+0000 7f19ffe00700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 0x7f19e40b4fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19ec013e40 tx=0x7f19ec0092d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.743+0000 7f1a15fb5b80 1 Processor -- start 2024-05-27T07:11:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.743+0000 7f1a15fb5b80 1 -- start start 2024-05-27T07:11:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae7f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce8adace40 con 0x55ce8ae84d30 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce8ad9c9e0 con 0x55ce8ae7edb0 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce8ada0a20 con 0x55ce8acffe40 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:49380/0 (socket says 172.21.15.16:49380) 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.744+0000 7f1a04a00700 1 -- 172.21.15.16:0/301898885 learned_addr learned my addr 172.21.15.16:0/301898885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 -- 172.21.15.16:0/301898885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 msgr2=0x55ce8ae7f1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae7f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 -- 172.21.15.16:0/301898885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 msgr2=0x55ce8ae7e870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae7e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 -- 172.21.15.16:0/301898885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8ad65420 con 0x55ce8acffe40 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.746+0000 7f1a04a00700 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a52e0 tx=0x7f19f00a9160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a06821f5a7a2524 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.747+0000 7f19f6000700 1 -- 172.21.15.16:0/301898885 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0021070 con 0x55ce8acffe40 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.747+0000 7f19f6000700 1 -- 172.21.15.16:0/301898885 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f00ad610 con 0x55ce8acffe40 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.747+0000 7f19f6000700 1 -- 172.21.15.16:0/301898885 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f00b1e30 con 0x55ce8acffe40 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/301898885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 msgr2=0x55ce8ae847f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a52e0 tx=0x7f19f00a9160 comp rx=0 tx=0).stop 2024-05-27T07:11:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/301898885 shutdown_connections 2024-05-27T07:11:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae7f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae847f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/301898885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae7e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.748+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/301898885 >> 172.21.15.16:0/301898885 conn(0x55ce8ad5ffc0 msgr2=0x55ce8ae798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.749+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/301898885 shutdown_connections 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.749+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/301898885 wait complete. 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.750+0000 7f1a15fb5b80 1 -- start start 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.750+0000 7f1a15fb5b80 1 Processor -- start 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae88b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8ae89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce8aca2fa0 con 0x55ce8ae7edb0 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce8ad3d760 con 0x55ce8acffe40 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce8ae8b800 con 0x55ce8ae84d30 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.752+0000 7f1a15fb5b80 10 client.0 _ll_register_callbacks cb 0x55ce8ad46c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.753+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.753+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:46258/0 (socket says 172.21.15.16:46258) 2024-05-27T07:11:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.753+0000 7f1a04a00700 1 -- 172.21.15.16:0/2589586306 learned_addr learned my addr 172.21.15.16:0/2589586306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:30.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.753+0000 7f19ffe00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae88b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f1a05400700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8ae89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f19ffe00700 1 -- 172.21.15.16:0/2589586306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 msgr2=0x55ce8ae89050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f19ffe00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8ae89050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f19ffe00700 1 -- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 msgr2=0x55ce8ae885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f19ffe00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f19ffe00700 1 -- 172.21.15.16:0/2589586306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8acec8d0 con 0x55ce8ae7edb0 2024-05-27T07:11:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.754+0000 7f1a04a00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:11:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.759+0000 7f19eaa00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:11:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.760+0000 7f19ffe00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae88b10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009c70 tx=0x7f19ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c17b4ac805046e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.761+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec035020 con 0x55ce8ae7edb0 2024-05-27T07:11:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.761+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7f19ec013550 con 0x55ce8ae7edb0 2024-05-27T07:11:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.762+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f00ad610 con 0x55ce8ae7edb0 2024-05-27T07:11:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.764+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19ec013750 con 0x55ce8ae7edb0 2024-05-27T07:11:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.764+0000 7f1a15fb5b80 10 client.16416 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.764+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ce8ad6d060 con 0x55ce8ae7edb0 2024-05-27T07:11:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.765+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f00ad610 con 0x55ce8ae7edb0 2024-05-27T07:11:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.765+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f19ec027bf0 con 0x55ce8ae7edb0 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.766+0000 7f1a15fb5b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.766+0000 7f1a15fb5b80 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.766+0000 7f1a15fb5b80 20 client.16416 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.767+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8ae98760 0x55ce8ae9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.767+0000 7f1a15fb5b80 4 client.16416 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.767+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce8adaed90 con 0x55ce8ae98760 2024-05-27T07:11:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.768+0000 7f1a04a00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8ae98760 0x55ce8ae9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.769+0000 7f1a04a00700 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8ae98760 0x55ce8ae9ab80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a273255ce169d60f server_cookie=3704fa9af3ae7d8a in_seq=0 out_seq=0 2024-05-27T07:11:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.769+0000 7f19eaa00700 10 client.16416 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.770+0000 7f19eaa00700 1 -- 172.21.15.16:0/2589586306 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ce8adaed90 con 0x55ce8ae98760 2024-05-27T07:11:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.770+0000 7f19eaa00700 10 client.16416 handle_command_reply: tid=0 2024-05-27T07:11:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.770+0000 7f1a15fb5b80 1 client.16416 shutdown 2024-05-27T07:11:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 msgr2=0x55ce8ae88b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae88b10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009c70 tx=0x7f19ec000bc0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 shutdown_connections 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8ae98760 0x55ce8ae9ab80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8ae89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8acffe40 0x55ce8ae885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/2589586306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae88b10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.771+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 >> 172.21.15.16:0/2589586306 conn(0x55ce8ad5ffc0 msgr2=0x55ce8ae79580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.772+0000 7f19eaa00700 0 client.16416 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.772+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 shutdown_connections 2024-05-27T07:11:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.772+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/2589586306 wait complete. 2024-05-27T07:11:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.774+0000 7f1a15fb5b80 20 client.16416 trim_cache size 0 max 16384 2024-05-27T07:11:30.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.824+0000 7f1a15fb5b80 1 Processor -- start 2024-05-27T07:11:30.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 -- start start 2024-05-27T07:11:30.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b6c30 con 0x55ce8ae84d30 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b8770 con 0x55ce8ae7edb0 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.826+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b7300 con 0x55ce8acffe40 2024-05-27T07:11:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.827+0000 7f1a05400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.827+0000 7f1a05400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46440/0 (socket says 172.21.15.16:46440) 2024-05-27T07:11:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.827+0000 7f1a05400700 1 -- 172.21.15.16:0/477750303 learned_addr learned my addr 172.21.15.16:0/477750303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.827+0000 7f19ffe00700 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a04a00700 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae7e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a05400700 1 -- 172.21.15.16:0/477750303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 msgr2=0x55ce8ae7e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a05400700 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae7e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a05400700 1 -- 172.21.15.16:0/477750303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 msgr2=0x55ce8ae8aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a05400700 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae8aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.828+0000 7f1a05400700 1 -- 172.21.15.16:0/477750303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8ae7a6d0 con 0x55ce8ae84d30 2024-05-27T07:11:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.829+0000 7f1a05400700 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007850 tx=0x7f19f8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d31f2375f8f589 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.830+0000 7f19f5600700 1 -- 172.21.15.16:0/477750303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f801a070 con 0x55ce8ae84d30 2024-05-27T07:11:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.830+0000 7f19f5600700 1 -- 172.21.15.16:0/477750303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f8004db0 con 0x55ce8ae84d30 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.830+0000 7f19f5600700 1 -- 172.21.15.16:0/477750303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8005580 con 0x55ce8ae84d30 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.832+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/477750303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 msgr2=0x55ce8ae82c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.832+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007850 tx=0x7f19f8009230 comp rx=0 tx=0).stop 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.832+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/477750303 shutdown_connections 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.833+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8ae84d30 0x55ce8ae82c40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.833+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8acffe40 0x55ce8ae7e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.833+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/477750303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.833+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/477750303 >> 172.21.15.16:0/477750303 conn(0x55ce8ad5ffc0 msgr2=0x55ce8ae93850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.834+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/477750303 shutdown_connections 2024-05-27T07:11:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.834+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/477750303 wait complete. 2024-05-27T07:11:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.837+0000 7f1a15fb5b80 1 -- start start 2024-05-27T07:11:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.837+0000 7f1a15fb5b80 1 Processor -- start 2024-05-27T07:11:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.838+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 0x55ce8ae91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.838+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.838+0000 7f1a15fb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.840+0000 7f19ffe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b80a0 con 0x55ce8acffe40 2024-05-27T07:11:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b79d0 con 0x55ce8ae7edb0 2024-05-27T07:11:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a15fb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b4c20 con 0x55ce8ae84d30 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a05400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a05400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:49410/0 (socket says 172.21.15.16:49410) 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a05400700 1 -- 172.21.15.16:0/152549794 learned_addr learned my addr 172.21.15.16:0/152549794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f1a15fb5b80 10 client.0 _ll_register_callbacks cb 0x55ce8ae842e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.841+0000 7f19ffe00700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae9fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:46282/0 (socket says 172.21.15.16:46282) 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a05400700 1 -- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 msgr2=0x55ce8ae9fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a05400700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae9fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a05400700 1 -- 172.21.15.16:0/152549794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 msgr2=0x55ce8ae91230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a04a00700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 0x55ce8ae91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a05400700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 0x55ce8ae91230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.842+0000 7f1a05400700 1 -- 172.21.15.16:0/152549794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8aea0a10 con 0x55ce8ae84d30 2024-05-27T07:11:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.843+0000 7f1a04a00700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 0x55ce8ae91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:11:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.843+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f8004db0 con 0x55ce8ae84d30 2024-05-27T07:11:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.845+0000 7f19eaa00700 10 client.26248 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.846+0000 7f1a05400700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19f8016d40 tx=0x7f19f8016d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2dd0e75b24f02b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.849+0000 7f19eaa00700 1 -- 172.21.15.16:0/152549794 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f802b060 con 0x55ce8ae84d30 2024-05-27T07:11:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.849+0000 7f19eaa00700 1 -- 172.21.15.16:0/152549794 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9613+0+0 (secure 0 0 0) 0x7f19f80051c0 con 0x55ce8ae84d30 2024-05-27T07:11:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.850+0000 7f19eaa00700 1 -- 172.21.15.16:0/152549794 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19f801a040 con 0x55ce8ae84d30 2024-05-27T07:11:30.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.851+0000 7f1a15fb5b80 10 client.26248 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:30.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.851+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55ce8af030c0 con 0x55ce8ae84d30 2024-05-27T07:11:30.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.854+0000 7f19eaa00700 1 -- 172.21.15.16:0/152549794 <== mon.1 v2:172.21.15.164:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f19f8005420 con 0x55ce8ae84d30 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 20 client.26248 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8af035b0 0x55ce8af059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 4 client.26248 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:11:30.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce8ada8570 con 0x55ce8af035b0 2024-05-27T07:11:30.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.856+0000 7f1a04a00700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8af035b0 0x55ce8af059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:30.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.858+0000 7f19eaa00700 10 client.26248 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:30.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.858+0000 7f1a04a00700 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8af035b0 0x55ce8af059d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3727ccdf4f842ce server_cookie=caafed951d714c77 in_seq=0 out_seq=0 2024-05-27T07:11:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f19eaa00700 1 -- 172.21.15.16:0/152549794 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce8ada8570 con 0x55ce8af035b0 2024-05-27T07:11:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f19eaa00700 10 client.26248 handle_command_reply: tid=0 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 client.26248 shutdown 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 msgr2=0x55ce8aea0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19f8016d40 tx=0x7f19f8016d70 comp rx=0 tx=0).stop 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 shutdown_connections 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ce8af035b0 0x55ce8af059d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce8ae84d30 0x55ce8aea0220 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce8acffe40 0x55ce8ae91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f19eaa00700 0 client.26248 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 --2- 172.21.15.16:0/152549794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce8ae7edb0 0x55ce8ae9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.859+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 >> 172.21.15.16:0/152549794 conn(0x55ce8ad5ffc0 msgr2=0x55ce8ae829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.860+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 shutdown_connections 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.860+0000 7f1a15fb5b80 1 -- 172.21.15.16:0/152549794 wait complete. 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.860+0000 7f1a15fb5b80 20 client.26248 trim_cache size 0 max 16384 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:11:30.852 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:11:30.853 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:11:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.861+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 msgr2=0x7f19e40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.861+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 0x7f19e40b4fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19ec013e40 tx=0x7f19ec0092d0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.861+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 msgr2=0x7f1a0006f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.861+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 0x7f1a0006f240 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dc60 comp rx=0 tx=0).stop 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 shutdown_connections 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f19e40b2aa0 0x7f19e40b4fb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a00109860 0x7f1a000749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a001045e0 0x7f1a0006f240 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 --2- 172.21.15.16:0/1323195131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a00108f10 0x7f1a0006f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.862+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 >> 172.21.15.16:0/1323195131 conn(0x7f1a00100030 msgr2=0x7f1a00108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:30.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.864+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 shutdown_connections 2024-05-27T07:11:30.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:30.864+0000 7f1a07400700 1 -- 172.21.15.16:0/1323195131 wait complete. 2024-05-27T07:11:31.945 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:11:31.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:11:32.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:31 smithi016 ceph-mon[31415]: pgmap v330: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 364 op/s 2024-05-27T07:11:32.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:31 smithi069 ceph-mon[35276]: pgmap v330: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 364 op/s 2024-05-27T07:11:32.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:31 smithi164 ceph-mon[35327]: pgmap v330: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 364 op/s 2024-05-27T07:11:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:11:34.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:33 smithi016 ceph-mon[31415]: pgmap v331: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 493 op/s 2024-05-27T07:11:34.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:34 smithi016 ceph-mon[31415]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:11:34.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:33 smithi069 ceph-mon[35276]: pgmap v331: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 493 op/s 2024-05-27T07:11:34.119 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:33 smithi069 ceph-mon[35276]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:11:34.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:33 smithi164 ceph-mon[35327]: pgmap v331: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 493 op/s 2024-05-27T07:11:34.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:33 smithi164 ceph-mon[35327]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:11:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:35 smithi016 ceph-mon[31415]: pgmap v333: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 446 op/s 2024-05-27T07:11:36.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:35 smithi069 ceph-mon[35276]: pgmap v333: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 446 op/s 2024-05-27T07:11:36.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:35 smithi164 ceph-mon[35327]: pgmap v333: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 446 op/s 2024-05-27T07:11:38.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:37 smithi069 ceph-mon[35276]: pgmap v334: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 493 op/s 2024-05-27T07:11:38.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:37 smithi164 ceph-mon[35327]: pgmap v334: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 493 op/s 2024-05-27T07:11:38.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:37 smithi016 ceph-mon[31415]: pgmap v334: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 24 MiB/s wr, 493 op/s 2024-05-27T07:11:40.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:39 smithi016 ceph-mon[31415]: pgmap v335: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 485 op/s 2024-05-27T07:11:40.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:39 smithi164 ceph-mon[35327]: pgmap v335: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 485 op/s 2024-05-27T07:11:40.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:39 smithi069 ceph-mon[35276]: pgmap v335: 129 pgs: 129 active+clean; 19 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 485 op/s 2024-05-27T07:11:42.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:41 smithi016 ceph-mon[31415]: pgmap v336: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 24 MiB/s wr, 498 op/s 2024-05-27T07:11:42.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:41 smithi164 ceph-mon[35327]: pgmap v336: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 24 MiB/s wr, 498 op/s 2024-05-27T07:11:42.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:41 smithi069 ceph-mon[35276]: pgmap v336: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 24 MiB/s wr, 498 op/s 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.408+0000 7fa076c00700 1 -- 172.21.15.16:0/1783374626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa070110f90 msgr2=0x7fa0701188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.408+0000 7fa076c00700 1 --2- 172.21.15.16:0/1783374626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa070110f90 0x7fa0701188f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b0d0 tx=0x7fa06800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.409+0000 7fa076c00700 1 -- 172.21.15.16:0/1783374626 shutdown_connections 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.409+0000 7fa076c00700 1 --2- 172.21.15.16:0/1783374626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa070110f90 0x7fa0701188f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.409+0000 7fa076c00700 1 --2- 172.21.15.16:0/1783374626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07010c820 0x7fa07010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.409+0000 7fa076c00700 1 --2- 172.21.15.16:0/1783374626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa070110640 0x7fa070110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.409+0000 7fa076c00700 1 -- 172.21.15.16:0/1783374626 >> 172.21.15.16:0/1783374626 conn(0x7fa07006f980 msgr2=0x7fa07006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.410+0000 7fa076c00700 1 -- 172.21.15.16:0/1783374626 shutdown_connections 2024-05-27T07:11:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.411+0000 7fa076c00700 1 -- 172.21.15.16:0/1783374626 wait complete. 2024-05-27T07:11:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.412+0000 7fa076c00700 1 Processor -- start 2024-05-27T07:11:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.413+0000 7fa076c00700 1 -- start start 2024-05-27T07:11:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.414+0000 7fa076c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa07010c820 0x7fa0701a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa076c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa070110640 0x7fa0701a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa076c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa076c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa07011bad0 con 0x7fa07010c820 2024-05-27T07:11:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa076c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa07011b950 con 0x7fa070110640 2024-05-27T07:11:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa076c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa07011bc50 con 0x7fa070110f90 2024-05-27T07:11:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa074c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa074c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:41728/0 (socket says 172.21.15.16:41728) 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.415+0000 7fa074c00700 1 -- 172.21.15.16:0/3368064118 learned_addr learned my addr 172.21.15.16:0/3368064118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.416+0000 7fa074c00700 1 -- 172.21.15.16:0/3368064118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa070110640 msgr2=0x7fa0701a6380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.416+0000 7fa074c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa070110640 0x7fa0701a6380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.416+0000 7fa074c00700 1 -- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa07010c820 msgr2=0x7fa0701a5e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.419+0000 7fa06fe00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa07010c820 0x7fa0701a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.419+0000 7fa074c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa07010c820 0x7fa0701a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.419+0000 7fa074c00700 1 -- 172.21.15.16:0/3368064118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0701a17c0 con 0x7fa070110f90 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.420+0000 7fa074c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa068004fb0 tx=0x7fa0680095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.420+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068021070 con 0x7fa070110f90 2024-05-27T07:11:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.421+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa068007d00 con 0x7fa070110f90 2024-05-27T07:11:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.421+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0701a1a50 con 0x7fa070110f90 2024-05-27T07:11:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.421+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701bebb0 con 0x7fa070110f90 2024-05-27T07:11:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.422+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680052a0 con 0x7fa070110f90 2024-05-27T07:11:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.424+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c005330 con 0x7fa070110f90 2024-05-27T07:11:43.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.427+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa06801d450 con 0x7fa070110f90 2024-05-27T07:11:43.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.433+0000 7fa06d600700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 0x7fa0540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:43.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.434+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fa0680e3ae0 con 0x7fa070110f90 2024-05-27T07:11:43.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.435+0000 7fa06fe00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 0x7fa0540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:43.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.437+0000 7fa06fe00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 0x7fa0540b5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa07006bad0 tx=0x7fa060009260 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:43.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.439+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0680ab160 con 0x7fa070110f90 2024-05-27T07:11:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.987+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa04c005620 con 0x7fa070110f90 2024-05-27T07:11:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.988+0000 7fa06d600700 1 -- 172.21.15.16:0/3368064118 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7fa0680bfe20 con 0x7fa070110f90 2024-05-27T07:11:43.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:11:43.984 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 msgr2=0x7fa0540b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 0x7fa0540b5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa07006bad0 tx=0x7fa060009260 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 msgr2=0x7fa0701a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa068004fb0 tx=0x7fa0680095a0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 shutdown_connections 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0540b2b40 0x7fa0540b5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa070110f90 0x7fa0701a0fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa07010c820 0x7fa0701a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 --2- 172.21.15.16:0/3368064118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa070110640 0x7fa0701a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 >> 172.21.15.16:0/3368064118 conn(0x7fa07006f980 msgr2=0x7fa070072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 shutdown_connections 2024-05-27T07:11:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:43.993+0000 7fa076c00700 1 -- 172.21.15.16:0/3368064118 wait complete. 2024-05-27T07:11:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:11:44.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:43 smithi164 ceph-mon[35327]: pgmap v337: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:11:44.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:43 smithi016 ceph-mon[31415]: pgmap v337: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:11:44.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:43 smithi069 ceph-mon[35276]: pgmap v337: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 501 op/s 2024-05-27T07:11:45.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3368064118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:11:45.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:11:45.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3368064118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:11:45.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3368064118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:11:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:11:46.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:45 smithi164 ceph-mon[35327]: pgmap v338: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 487 op/s 2024-05-27T07:11:46.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:45 smithi016 ceph-mon[31415]: pgmap v338: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 487 op/s 2024-05-27T07:11:46.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:45 smithi069 ceph-mon[35276]: pgmap v338: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 487 op/s 2024-05-27T07:11:47.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:46 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:11:47.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:46 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:11:47.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:46 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:11:48.226 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:47 smithi164 ceph-mon[35327]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 551 op/s 2024-05-27T07:11:48.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:47 smithi016 ceph-mon[31415]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 551 op/s 2024-05-27T07:11:48.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:47 smithi069 ceph-mon[35276]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 551 op/s 2024-05-27T07:11:49.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:48 smithi164 ceph-mon[35327]: pgmap v340: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 456 op/s 2024-05-27T07:11:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:48 smithi016 ceph-mon[31415]: pgmap v340: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 456 op/s 2024-05-27T07:11:49.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:48 smithi069 ceph-mon[35276]: pgmap v340: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 456 op/s 2024-05-27T07:11:51.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:51 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:51 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:51 smithi016 ceph-mon[31415]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 455 op/s 2024-05-27T07:11:51.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:51 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:51 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:51 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:51 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:51 smithi164 ceph-mon[35327]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 455 op/s 2024-05-27T07:11:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:51 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:51 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:51 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:51 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:51 smithi069 ceph-mon[35276]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 455 op/s 2024-05-27T07:11:51.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:51 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:51.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:51 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:11:52.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:11:52.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:11:52.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:52 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:11:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:11:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:11:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:52 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:11:52.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi164 to 176.4M 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-05-27T07:11:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:52 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi164 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-27T07:11:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:53 smithi164 ceph-mon[35327]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 557 op/s 2024-05-27T07:11:53.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:53 smithi016 ceph-mon[31415]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 557 op/s 2024-05-27T07:11:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:53 smithi069 ceph-mon[35276]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 557 op/s 2024-05-27T07:11:55.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:55 smithi016 ceph-mon[31415]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 416 op/s 2024-05-27T07:11:55.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:55 smithi164 ceph-mon[35327]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 416 op/s 2024-05-27T07:11:55.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:55 smithi069 ceph-mon[35276]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 416 op/s 2024-05-27T07:11:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 -- 172.21.15.16:0/3310893806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 msgr2=0x7f74001036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 --2- 172.21.15.16:0/3310893806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f74001036e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b0d0 tx=0x7f73f800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 -- 172.21.15.16:0/3310893806 shutdown_connections 2024-05-27T07:11:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 --2- 172.21.15.16:0/3310893806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 0x7f740010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 --2- 172.21.15.16:0/3310893806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f74001036e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 --2- 172.21.15.16:0/3310893806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7400107c00 0x7f7400108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.210+0000 7f7406200700 1 -- 172.21.15.16:0/3310893806 >> 172.21.15.16:0/3310893806 conn(0x7f7400078ab0 msgr2=0x7f7400078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.211+0000 7f7406200700 1 -- 172.21.15.16:0/3310893806 shutdown_connections 2024-05-27T07:11:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.211+0000 7f7406200700 1 -- 172.21.15.16:0/3310893806 wait complete. 2024-05-27T07:11:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.212+0000 7f7406200700 1 Processor -- start 2024-05-27T07:11:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.215+0000 7f7406200700 1 -- start start 2024-05-27T07:11:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7406200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7406200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7400107c00 0x7f7400199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7404e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7404e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:53912/0 (socket says 172.21.15.16:53912) 2024-05-27T07:11:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7404e00700 1 -- 172.21.15.16:0/29524790 learned_addr learned my addr 172.21.15.16:0/29524790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.217+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 0x7f740019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.218+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74001130f0 con 0x7f7400107c00 2024-05-27T07:11:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.218+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7400112f70 con 0x7f7400108550 2024-05-27T07:11:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.218+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7400113270 con 0x7f74001032d0 2024-05-27T07:11:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.218+0000 7f7405800700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 0x7f740019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.219+0000 7f7404e00700 1 -- 172.21.15.16:0/29524790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 msgr2=0x7f740019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.219+0000 7f7404e00700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 0x7f740019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.219+0000 7f7404e00700 1 -- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7400107c00 msgr2=0x7f7400199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.219+0000 7f7404e00700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7400107c00 0x7f7400199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.220+0000 7f7404e00700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740019d7f0 con 0x7f74001032d0 2024-05-27T07:11:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.220+0000 7f7404e00700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73f8007dc0 tx=0x7f73f8007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.221+0000 7f73fd600700 1 -- 172.21.15.16:0/29524790 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f801f030 con 0x7f74001032d0 2024-05-27T07:11:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.221+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740019da80 con 0x7f74001032d0 2024-05-27T07:11:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.221+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740019e010 con 0x7f74001032d0 2024-05-27T07:11:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.221+0000 7f73fd600700 1 -- 172.21.15.16:0/29524790 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73f80075c0 con 0x7f74001032d0 2024-05-27T07:11:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.221+0000 7f73fd600700 1 -- 172.21.15.16:0/29524790 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f801f030 con 0x7f74001032d0 2024-05-27T07:11:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.233+0000 7f73fd600700 1 -- 172.21.15.16:0/29524790 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f73f8022030 con 0x7f74001032d0 2024-05-27T07:11:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.235+0000 7f73fd600700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 0x7f73e40b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.236+0000 7f73fd600700 1 -- 172.21.15.16:0/29524790 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f73f800f020 con 0x7f74001032d0 2024-05-27T07:11:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.238+0000 7f73ffe00700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 0x7f73e40b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.252+0000 7f73ffe00700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 0x7f73e40b5180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f74001096a0 tx=0x7f73f0006d20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.254+0000 7f74162f9b80 1 Processor -- start 2024-05-27T07:11:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.257+0000 7f74162f9b80 1 -- start start 2024-05-27T07:11:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2545750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d23c5e40 0x5555d2545c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555d2472e40 con 0x5555d24b19f0 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5555d24629e0 con 0x5555d23c5e40 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.258+0000 7f74162f9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5555d2466a20 con 0x5555d254f2c0 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.259+0000 7f7405800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.259+0000 7f7404e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2545750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.259+0000 7f7405800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:53924/0 (socket says 172.21.15.16:53924) 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.259+0000 7f7405800700 1 -- 172.21.15.16:0/3424123309 learned_addr learned my addr 172.21.15.16:0/3424123309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.260+0000 7f7405800700 1 -- 172.21.15.16:0/3424123309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d23c5e40 msgr2=0x5555d2545c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.260+0000 7f7405800700 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d23c5e40 0x5555d2545c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.260+0000 7f7405800700 1 -- 172.21.15.16:0/3424123309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 msgr2=0x5555d2545750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.260+0000 7f7405800700 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2545750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.260+0000 7f7405800700 1 -- 172.21.15.16:0/3424123309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d242b420 con 0x5555d254f2c0 2024-05-27T07:11:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.261+0000 7f7405800700 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73e800acb0 tx=0x7f73e800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265b944b735c2af8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.261+0000 7f73f6000700 1 -- 172.21.15.16:0/3424123309 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e800fe50 con 0x5555d254f2c0 2024-05-27T07:11:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.261+0000 7f73f6000700 1 -- 172.21.15.16:0/3424123309 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73e8004cc0 con 0x5555d254f2c0 2024-05-27T07:11:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.261+0000 7f73f6000700 1 -- 172.21.15.16:0/3424123309 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8018440 con 0x5555d254f2c0 2024-05-27T07:11:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.262+0000 7f74162f9b80 1 -- 172.21.15.16:0/3424123309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 msgr2=0x5555d25516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.262+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73e800acb0 tx=0x7f73e800f3c0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.263+0000 7f74162f9b80 1 -- 172.21.15.16:0/3424123309 shutdown_connections 2024-05-27T07:11:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.263+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d254f2c0 0x5555d25516e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.263+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2545750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.263+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3424123309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d23c5e40 0x5555d2545c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.263+0000 7f74162f9b80 1 -- 172.21.15.16:0/3424123309 >> 172.21.15.16:0/3424123309 conn(0x5555d245f390 msgr2=0x5555d2540710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.264+0000 7f74162f9b80 1 -- 172.21.15.16:0/3424123309 shutdown_connections 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.264+0000 7f74162f9b80 1 -- 172.21.15.16:0/3424123309 wait complete. 2024-05-27T07:11:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.265+0000 7f74162f9b80 1 -- start start 2024-05-27T07:11:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.265+0000 7f74162f9b80 1 Processor -- start 2024-05-27T07:11:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.267+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d254c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.267+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d254c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.267+0000 7f7404e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d254c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d254cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f73ffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d254c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f7404e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d254c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34292/0 (socket says 172.21.15.16:34292) 2024-05-27T07:11:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f7404e00700 1 -- 172.21.15.16:0/3910159219 learned_addr learned my addr 172.21.15.16:0/3910159219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555d2368fa0 con 0x5555d23c5e40 2024-05-27T07:11:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5555d2403760 con 0x5555d254f2c0 2024-05-27T07:11:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.268+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5555d2551e00 con 0x5555d24b19f0 2024-05-27T07:11:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f74162f9b80 10 client.0 _ll_register_callbacks cb 0x5555d240cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f7405800700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d254cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f73ffe00700 1 -- 172.21.15.16:0/3910159219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 msgr2=0x5555d254cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d254cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f73ffe00700 1 -- 172.21.15.16:0/3910159219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 msgr2=0x5555d254c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d254c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.269+0000 7f73ffe00700 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d23b28d0 con 0x5555d24b19f0 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.270+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d254c730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7400109810 tx=0x7f73f0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c817fc5c1d30931 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.270+0000 7f73eea00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.270+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0023060 con 0x5555d24b19f0 2024-05-27T07:11:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.271+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f73f0012b50 con 0x5555d24b19f0 2024-05-27T07:11:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.271+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e8004cc0 con 0x5555d24b19f0 2024-05-27T07:11:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.271+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f73f002a050 con 0x5555d24b19f0 2024-05-27T07:11:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.271+0000 7f74162f9b80 10 client.26266 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.272+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5555d2433060 con 0x5555d24b19f0 2024-05-27T07:11:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.273+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73e8004cc0 con 0x5555d24b19f0 2024-05-27T07:11:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.275+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f73f0013c60 con 0x5555d24b19f0 2024-05-27T07:11:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.277+0000 7f74162f9b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.277+0000 7f74162f9b80 10 client.26266 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:11:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.277+0000 7f74162f9b80 20 client.26266 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.278+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d255ebb0 0x5555d2560fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.278+0000 7f74162f9b80 4 client.26266 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:11:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.278+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d2474d90 con 0x5555d255ebb0 2024-05-27T07:11:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.281+0000 7f7404e00700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d255ebb0 0x5555d2560fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.283+0000 7f7404e00700 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d255ebb0 0x5555d2560fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=998ba273d680ba93 server_cookie=41aea000d85bcb9d in_seq=0 out_seq=0 2024-05-27T07:11:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.283+0000 7f73eea00700 10 client.26266 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:11:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.284+0000 7f73eea00700 1 -- 172.21.15.16:0/3910159219 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5555d2474d90 con 0x5555d255ebb0 2024-05-27T07:11:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.284+0000 7f73eea00700 10 client.26266 handle_command_reply: tid=0 2024-05-27T07:11:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.284+0000 7f74162f9b80 1 client.26266 shutdown 2024-05-27T07:11:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 msgr2=0x5555d254c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d254c730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7400109810 tx=0x7f73f0004860 comp rx=0 tx=0).stop 2024-05-27T07:11:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 shutdown_connections 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d255ebb0 0x5555d2560fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d254cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d254c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3910159219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d254c730 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f73eea00700 0 client.26266 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:11:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 >> 172.21.15.16:0/3910159219 conn(0x5555d245f390 msgr2=0x5555d2540300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 shutdown_connections 2024-05-27T07:11:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.286+0000 7f74162f9b80 1 -- 172.21.15.16:0/3910159219 wait complete. 2024-05-27T07:11:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.288+0000 7f74162f9b80 20 client.26266 trim_cache size 0 max 16384 2024-05-27T07:11:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.332+0000 7f74162f9b80 1 Processor -- start 2024-05-27T07:11:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 -- start start 2024-05-27T07:11:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2547240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2547780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00ee90 con 0x5555d24b19f0 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc0109d0 con 0x5555d254f2c0 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.333+0000 7f74162f9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00f560 con 0x5555d23c5e40 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7404e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7404e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:53942/0 (socket says 172.21.15.16:53942) 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7404e00700 1 -- 172.21.15.16:0/466298661 learned_addr learned my addr 172.21.15.16:0/466298661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7405800700 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2547780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7404e00700 1 -- 172.21.15.16:0/466298661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 msgr2=0x5555d2547780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.334+0000 7f7404e00700 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2547780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.335+0000 7f7404e00700 1 -- 172.21.15.16:0/466298661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 msgr2=0x5555d2547240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.335+0000 7f7404e00700 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2547240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.335+0000 7f7404e00700 1 -- 172.21.15.16:0/466298661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d254a760 con 0x5555d23c5e40 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.335+0000 7f7404e00700 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009fd0 tx=0x7f73f8093040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdc74dc858a5500 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.336+0000 7f73f5600700 1 -- 172.21.15.16:0/466298661 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f80b43f0 con 0x5555d23c5e40 2024-05-27T07:11:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.336+0000 7f73f5600700 1 -- 172.21.15.16:0/466298661 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73f80e1c80 con 0x5555d23c5e40 2024-05-27T07:11:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.336+0000 7f73f5600700 1 -- 172.21.15.16:0/466298661 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f80b43f0 con 0x5555d23c5e40 2024-05-27T07:11:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.337+0000 7f74162f9b80 1 -- 172.21.15.16:0/466298661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 msgr2=0x5555d255af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.337+0000 7f74162f9b80 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009fd0 tx=0x7f73f8093040 comp rx=0 tx=0).stop 2024-05-27T07:11:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 -- 172.21.15.16:0/466298661 shutdown_connections 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2547780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d23c5e40 0x5555d255af30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 --2- 172.21.15.16:0/466298661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d24b19f0 0x5555d2547240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 -- 172.21.15.16:0/466298661 >> 172.21.15.16:0/466298661 conn(0x5555d245f390 msgr2=0x5555d254aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.338+0000 7f74162f9b80 1 -- 172.21.15.16:0/466298661 shutdown_connections 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.339+0000 7f74162f9b80 1 -- 172.21.15.16:0/466298661 wait complete. 2024-05-27T07:11:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.340+0000 7f74162f9b80 1 -- start start 2024-05-27T07:11:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.340+0000 7f74162f9b80 1 Processor -- start 2024-05-27T07:11:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.342+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d2557760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2553000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc010300 con 0x5555d23c5e40 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00fc30 con 0x5555d254f2c0 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00ce50 con 0x5555d24b19f0 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f74162f9b80 10 client.0 _ll_register_callbacks cb 0x5555d240cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f73ffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f73ffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:53956/0 (socket says 172.21.15.16:53956) 2024-05-27T07:11:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.343+0000 7f73ffe00700 1 -- 172.21.15.16:0/3162828808 learned_addr learned my addr 172.21.15.16:0/3162828808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 -- 172.21.15.16:0/3162828808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 msgr2=0x5555d2553000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2553000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 -- 172.21.15.16:0/3162828808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 msgr2=0x5555d2557760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d2557760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 -- 172.21.15.16:0/3162828808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d25536b0 con 0x5555d24b19f0 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73eea00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.344+0000 7f73ffe00700 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006e20 tx=0x7f73f00108d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d448032c368f07b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:11:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.345+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0012ce0 con 0x5555d24b19f0 2024-05-27T07:11:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.346+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f73f001b050 con 0x5555d24b19f0 2024-05-27T07:11:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.349+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f80e1c80 con 0x5555d24b19f0 2024-05-27T07:11:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.351+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f73f0012ce0 con 0x5555d24b19f0 2024-05-27T07:11:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.351+0000 7f74162f9b80 10 client.26272 fetch_fsmap learned FSMap version 31 2024-05-27T07:11:56.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.352+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5555d25c9490 con 0x5555d24b19f0 2024-05-27T07:11:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.352+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73f0033070 con 0x5555d24b19f0 2024-05-27T07:11:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f73f0010940 con 0x5555d24b19f0 2024-05-27T07:11:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 10 client.26272 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:11:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 10 client.26272 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:11:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 20 client.26272 populate_metadata read hostname 'smithi016' 2024-05-27T07:11:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d25c9bf0 0x5555d25cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:11:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 4 client.26272 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:11:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.354+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f73f80e1c80 con 0x5555d25c9bf0 2024-05-27T07:11:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.355+0000 7f7404e00700 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d25c9bf0 0x5555d25cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:11:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.356+0000 7f7404e00700 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d25c9bf0 0x5555d25cc010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=971b7eb2bfca9f49 server_cookie=43f20878fe4daa45 in_seq=0 out_seq=0 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.356+0000 7f73eea00700 10 client.26272 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.356+0000 7f73eea00700 1 -- 172.21.15.16:0/3162828808 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f73f80e1c80 con 0x5555d25c9bf0 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.356+0000 7f73eea00700 10 client.26272 handle_command_reply: tid=0 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.356+0000 7f74162f9b80 1 client.26272 shutdown 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 msgr2=0x5555d2552ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006e20 tx=0x7f73f00108d0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 shutdown_connections 2024-05-27T07:11:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x5555d25c9bf0 0x5555d25cc010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555d254f2c0 0x5555d2553000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555d23c5e40 0x5555d2557760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 --2- 172.21.15.16:0/3162828808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555d24b19f0 0x5555d2552ac0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f73eea00700 0 client.26272 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.357+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 >> 172.21.15.16:0/3162828808 conn(0x5555d245f390 msgr2=0x5555d2547240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.358+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 shutdown_connections 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.358+0000 7f74162f9b80 1 -- 172.21.15.16:0/3162828808 wait complete. 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.358+0000 7f74162f9b80 20 client.26272 trim_cache size 0 max 16384 2024-05-27T07:11:56.350 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.359+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 msgr2=0x7f73e40b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.359+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 0x7f73e40b5180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f74001096a0 tx=0x7f73f0006d20 comp rx=0 tx=0).stop 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.359+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 msgr2=0x7f7400198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.359+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73f8007dc0 tx=0x7f73f8007df0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 shutdown_connections 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f73e40b2c70 0x7f73e40b5180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7400108550 0x7f740019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74001032d0 0x7f7400198ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 --2- 172.21.15.16:0/29524790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7400107c00 0x7f7400199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:11:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 >> 172.21.15.16:0/29524790 conn(0x7f7400078ab0 msgr2=0x7f7400106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:11:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 shutdown_connections 2024-05-27T07:11:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:11:56.360+0000 7f7406200700 1 -- 172.21.15.16:0/29524790 wait complete. 2024-05-27T07:11:57.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:57 smithi164 ceph-mon[35327]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 529 op/s 2024-05-27T07:11:57.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:57 smithi016 ceph-mon[31415]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 529 op/s 2024-05-27T07:11:57.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:57 smithi069 ceph-mon[35276]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 529 op/s 2024-05-27T07:11:57.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:11:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:11:59.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:11:59 smithi016 ceph-mon[31415]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 21 MiB/s wr, 438 op/s 2024-05-27T07:11:59.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:11:59 smithi069 ceph-mon[35276]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 21 MiB/s wr, 438 op/s 2024-05-27T07:11:59.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:11:59 smithi164 ceph-mon[35327]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 21 MiB/s wr, 438 op/s 2024-05-27T07:12:01.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:01 smithi164 ceph-mon[35327]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 439 op/s 2024-05-27T07:12:01.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:01 smithi016 ceph-mon[31415]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 439 op/s 2024-05-27T07:12:01.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:01 smithi069 ceph-mon[35276]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 439 op/s 2024-05-27T07:12:02.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:02.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:02.584 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:02 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:02.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:02 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:02.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:02.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:02 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:03.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:03 smithi016 ceph-mon[31415]: pgmap v347: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 27 MiB/s wr, 546 op/s 2024-05-27T07:12:03.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:03 smithi164 ceph-mon[35327]: pgmap v347: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 27 MiB/s wr, 546 op/s 2024-05-27T07:12:03.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:03 smithi069 ceph-mon[35276]: pgmap v347: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 27 MiB/s wr, 546 op/s 2024-05-27T07:12:04.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:12:04.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:04.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:12:04.832 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:04.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: Unable to set osd_memory_target on smithi016 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:12:04.868 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:12:06.292 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:05 smithi016 ceph-mon[31415]: pgmap v348: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 419 op/s 2024-05-27T07:12:06.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:05 smithi164 ceph-mon[35327]: pgmap v348: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 419 op/s 2024-05-27T07:12:06.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:05 smithi069 ceph-mon[35276]: pgmap v348: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 419 op/s 2024-05-27T07:12:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 -- 172.21.15.16:0/4226845784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c108f10 msgr2=0x7f9e1c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 --2- 172.21.15.16:0/4226845784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c108f10 0x7f9e1c109320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400b0a0 tx=0x7f9e0400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 -- 172.21.15.16:0/4226845784 shutdown_connections 2024-05-27T07:12:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 --2- 172.21.15.16:0/4226845784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c109860 0x7f9e1c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 --2- 172.21.15.16:0/4226845784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 --2- 172.21.15.16:0/4226845784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c108f10 0x7f9e1c109320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.993+0000 7f9e23400700 1 -- 172.21.15.16:0/4226845784 >> 172.21.15.16:0/4226845784 conn(0x7f9e1c100030 msgr2=0x7f9e1c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:06.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.994+0000 7f9e23400700 1 -- 172.21.15.16:0/4226845784 shutdown_connections 2024-05-27T07:12:06.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.994+0000 7f9e23400700 1 -- 172.21.15.16:0/4226845784 wait complete. 2024-05-27T07:12:06.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.996+0000 7f9e23400700 1 Processor -- start 2024-05-27T07:12:06.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.996+0000 7f9e23400700 1 -- start start 2024-05-27T07:12:06.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.997+0000 7f9e23400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e23400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e23400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c109860 0x7f9e1c06e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e23400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1143e0 con 0x7f9e1c109860 2024-05-27T07:12:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e23400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c114260 con 0x7f9e1c1045e0 2024-05-27T07:12:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e23400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c114560 con 0x7f9e1c108f10 2024-05-27T07:12:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.998+0000 7f9e20a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:33228/0 (socket says 172.21.15.16:33228) 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e20a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51582/0 (socket says 172.21.15.16:51582) 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 -- 172.21.15.16:0/951754546 learned_addr learned my addr 172.21.15.16:0/951754546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e21400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c109860 0x7f9e1c06e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 -- 172.21.15.16:0/951754546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 msgr2=0x7f9e1c075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 -- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c109860 msgr2=0x7f9e1c06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c109860 0x7f9e1c06e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:06.999+0000 7f9e1be00700 1 -- 172.21.15.16:0/951754546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c06f140 con 0x7f9e1c108f10 2024-05-27T07:12:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.000+0000 7f9e1be00700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e04015000 tx=0x7f9e04009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.001+0000 7f9e19600700 1 -- 172.21.15.16:0/951754546 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04021070 con 0x7f9e1c108f10 2024-05-27T07:12:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.001+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c072dc0 con 0x7f9e1c108f10 2024-05-27T07:12:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.001+0000 7f9e19600700 1 -- 172.21.15.16:0/951754546 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e0401d8e0 con 0x7f9e1c108f10 2024-05-27T07:12:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.001+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c073300 con 0x7f9e1c108f10 2024-05-27T07:12:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.001+0000 7f9e19600700 1 -- 172.21.15.16:0/951754546 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04005400 con 0x7f9e1c108f10 2024-05-27T07:12:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.009+0000 7f9e19600700 1 -- 172.21.15.16:0/951754546 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9e04024070 con 0x7f9e1c108f10 2024-05-27T07:12:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.012+0000 7f9e19600700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 0x7f9e000b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.012+0000 7f9e19600700 1 -- 172.21.15.16:0/951754546 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f9e040e4fa0 con 0x7f9e1c108f10 2024-05-27T07:12:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.013+0000 7f9e20a00700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 0x7f9e000b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.028+0000 7f9e20a00700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 0x7f9e000b5000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c006fd0 tx=0x7f9e0c008040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.035+0000 7f9e32005b80 1 Processor -- start 2024-05-27T07:12:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.035+0000 7f9e32005b80 1 -- start start 2024-05-27T07:12:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c29111470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c29111dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c29035e40 con 0x562c2911a4e0 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c290259e0 con 0x562c291119b0 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.036+0000 7f9e32005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562c29029a20 con 0x562c28f88e40 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.037+0000 7f9e21400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.037+0000 7f9e21400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34840/0 (socket says 172.21.15.16:34840) 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.037+0000 7f9e21400700 1 -- 172.21.15.16:0/1256363045 learned_addr learned my addr 172.21.15.16:0/1256363045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.037+0000 7f9e20a00700 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c29111470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.037+0000 7f9e1be00700 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c29111dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 -- 172.21.15.16:0/1256363045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 msgr2=0x562c29111470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c29111470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 -- 172.21.15.16:0/1256363045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 msgr2=0x562c29111dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c29111dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 -- 172.21.15.16:0/1256363045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c28fee420 con 0x562c2911a4e0 2024-05-27T07:12:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.038+0000 7f9e21400700 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009c60 tx=0x7f9e14009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9383f0f32a3055 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.039+0000 7f9e12000700 1 -- 172.21.15.16:0/1256363045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010c30 con 0x562c2911a4e0 2024-05-27T07:12:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.039+0000 7f9e12000700 1 -- 172.21.15.16:0/1256363045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e14010dd0 con 0x562c2911a4e0 2024-05-27T07:12:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.039+0000 7f9e12000700 1 -- 172.21.15.16:0/1256363045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14012450 con 0x562c2911a4e0 2024-05-27T07:12:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.041+0000 7f9e32005b80 1 -- 172.21.15.16:0/1256363045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 msgr2=0x562c2911c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.041+0000 7f9e32005b80 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009c60 tx=0x7f9e14009f70 comp rx=0 tx=0).stop 2024-05-27T07:12:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.042+0000 7f9e32005b80 1 -- 172.21.15.16:0/1256363045 shutdown_connections 2024-05-27T07:12:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.042+0000 7f9e32005b80 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c2911c900 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.042+0000 7f9e32005b80 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c29111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.042+0000 7f9e32005b80 1 --2- 172.21.15.16:0/1256363045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c29111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.042+0000 7f9e32005b80 1 -- 172.21.15.16:0/1256363045 >> 172.21.15.16:0/1256363045 conn(0x562c28fe8fc0 msgr2=0x562c29102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.047+0000 7f9e32005b80 1 -- 172.21.15.16:0/1256363045 shutdown_connections 2024-05-27T07:12:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.048+0000 7f9e32005b80 1 -- 172.21.15.16:0/1256363045 wait complete. 2024-05-27T07:12:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.048+0000 7f9e32005b80 1 -- start start 2024-05-27T07:12:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.049+0000 7f9e32005b80 1 Processor -- start 2024-05-27T07:12:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.052+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 0x562c291174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.052+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c291179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.052+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 0x562c29117f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e32005b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c28f2bfa0 con 0x562c28f88e40 2024-05-27T07:12:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e32005b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c28fc6760 con 0x562c291119b0 2024-05-27T07:12:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e32005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562c2911cff0 con 0x562c2911a4e0 2024-05-27T07:12:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c291179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e32005b80 10 client.0 _ll_register_callbacks cb 0x562c28fcfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c291179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51612/0 (socket says 172.21.15.16:51612) 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.053+0000 7f9e1be00700 1 -- 172.21.15.16:0/4080105942 learned_addr learned my addr 172.21.15.16:0/4080105942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.054+0000 7f9e20a00700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 0x562c291174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.054+0000 7f9e21400700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 0x562c29117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.054+0000 7f9e21400700 1 -- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 msgr2=0x562c291179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.054+0000 7f9e21400700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c291179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e21400700 1 -- 172.21.15.16:0/4080105942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 msgr2=0x562c291174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e21400700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 0x562c291174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e21400700 1 -- 172.21.15.16:0/4080105942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c28f758d0 con 0x562c2911a4e0 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e20a00700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 0x562c291174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e21400700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 0x562c29117f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e14019000 tx=0x7f9e14005610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1387a2030d1cb38d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.055+0000 7f9e0aa00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:12:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.056+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1402a060 con 0x562c2911a4e0 2024-05-27T07:12:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.056+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f9e14004960 con 0x562c2911a4e0 2024-05-27T07:12:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.057+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e14010dd0 con 0x562c2911a4e0 2024-05-27T07:12:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.059+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e14004b60 con 0x562c2911a4e0 2024-05-27T07:12:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.060+0000 7f9e32005b80 10 client.26284 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.060+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c28ff6060 con 0x562c2911a4e0 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.064+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e14010dd0 con 0x562c2911a4e0 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.064+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9e1400f040 con 0x562c2911a4e0 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 10 client.26284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 20 client.26284 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c29129da0 0x562c2912c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 4 client.26284 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:12:07.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.065+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c29037d90 con 0x562c29129da0 2024-05-27T07:12:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.066+0000 7f9e20a00700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c29129da0 0x562c2912c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.069+0000 7f9e20a00700 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c29129da0 0x562c2912c1c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc197b57097e8dfe server_cookie=a881102b40976776 in_seq=0 out_seq=0 2024-05-27T07:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.069+0000 7f9e0aa00700 10 client.26284 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.072+0000 7f9e0aa00700 1 -- 172.21.15.16:0/4080105942 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562c29037d90 con 0x562c29129da0 2024-05-27T07:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.072+0000 7f9e0aa00700 10 client.26284 handle_command_reply: tid=0 2024-05-27T07:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.073+0000 7f9e32005b80 1 client.26284 shutdown 2024-05-27T07:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.073+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 msgr2=0x562c29117f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.073+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 0x562c29117f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e14019000 tx=0x7f9e14005610 comp rx=0 tx=0).stop 2024-05-27T07:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 shutdown_connections 2024-05-27T07:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c2911a4e0 0x562c29117f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c28f88e40 0x562c291174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c29129da0 0x562c2912c1c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 --2- 172.21.15.16:0/4080105942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c291119b0 0x562c291179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.074+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 >> 172.21.15.16:0/4080105942 conn(0x562c28fe8fc0 msgr2=0x562c291026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.075+0000 7f9e0aa00700 0 client.26284 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.075+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 shutdown_connections 2024-05-27T07:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.075+0000 7f9e32005b80 1 -- 172.21.15.16:0/4080105942 wait complete. 2024-05-27T07:12:07.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.076+0000 7f9e32005b80 20 client.26284 trim_cache size 0 max 16384 2024-05-27T07:12:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.111+0000 7f9e32005b80 1 Processor -- start 2024-05-27T07:12:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 -- start start 2024-05-27T07:12:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c2912edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c2911a4e0 0x562c29112480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b6a90 con 0x562c291119b0 2024-05-27T07:12:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b85d0 con 0x562c2911a4e0 2024-05-27T07:12:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.114+0000 7f9e32005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b7160 con 0x562c28f88e40 2024-05-27T07:12:07.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.118+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.118+0000 7f9e1be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34856/0 (socket says 172.21.15.16:34856) 2024-05-27T07:12:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.118+0000 7f9e1be00700 1 -- 172.21.15.16:0/3496025196 learned_addr learned my addr 172.21.15.16:0/3496025196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.119+0000 7f9e1be00700 1 -- 172.21.15.16:0/3496025196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 msgr2=0x562c2912edd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:12:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.119+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c2912edd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.119+0000 7f9e1be00700 1 -- 172.21.15.16:0/3496025196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c2911a4e0 msgr2=0x562c29112480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:12:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.119+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c2911a4e0 0x562c29112480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.119+0000 7f9e1be00700 1 -- 172.21.15.16:0/3496025196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c29108420 con 0x562c291119b0 2024-05-27T07:12:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.120+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e040abe00 tx=0x7f9e0400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eac366bfdcaa9c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.124+0000 7f9e11600700 1 -- 172.21.15.16:0/3496025196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0402b040 con 0x562c291119b0 2024-05-27T07:12:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.124+0000 7f9e11600700 1 -- 172.21.15.16:0/3496025196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e04009ca0 con 0x562c291119b0 2024-05-27T07:12:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.124+0000 7f9e11600700 1 -- 172.21.15.16:0/3496025196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040ac120 con 0x562c291119b0 2024-05-27T07:12:07.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 -- 172.21.15.16:0/3496025196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 msgr2=0x562c2912f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e040abe00 tx=0x7f9e0400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 -- 172.21.15.16:0/3496025196 shutdown_connections 2024-05-27T07:12:07.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c2911a4e0 0x562c29112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c28f88e40 0x562c2912edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3496025196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c291119b0 0x562c2912f310 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 -- 172.21.15.16:0/3496025196 >> 172.21.15.16:0/3496025196 conn(0x562c28fe8fc0 msgr2=0x562c291024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 -- 172.21.15.16:0/3496025196 shutdown_connections 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.126+0000 7f9e32005b80 1 -- 172.21.15.16:0/3496025196 wait complete. 2024-05-27T07:12:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.127+0000 7f9e32005b80 1 -- start start 2024-05-27T07:12:07.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.127+0000 7f9e32005b80 1 Processor -- start 2024-05-27T07:12:07.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.128+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c28f88e40 0x562c2912c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 0x562c2912c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b7f00 con 0x562c2911a4e0 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b7830 con 0x562c28f88e40 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1b4a50 con 0x562c291119b0 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e32005b80 10 client.0 _ll_register_callbacks cb 0x562c28fcfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.129+0000 7f9e21400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e21400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34862/0 (socket says 172.21.15.16:34862) 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e21400700 1 -- 172.21.15.16:0/3298210686 learned_addr learned my addr 172.21.15.16:0/3298210686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:07.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 0x562c2912c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e1be00700 1 -- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c28f88e40 msgr2=0x562c2912c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c28f88e40 0x562c2912c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.130+0000 7f9e1be00700 1 -- 172.21.15.16:0/3298210686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 msgr2=0x562c29192600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:07.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.131+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.131+0000 7f9e1be00700 1 -- 172.21.15.16:0/3298210686 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c29192b40 con 0x562c291119b0 2024-05-27T07:12:07.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.131+0000 7f9e21400700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.132+0000 7f9e1be00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 0x562c2912c790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e040b4e50 tx=0x7f9e040ab940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17366d360f9e8d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.133+0000 7f9e0aa00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.133+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0402b070 con 0x562c291119b0 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.133+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f9e040ac320 con 0x562c291119b0 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.133+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e04009ca0 con 0x562c291119b0 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.134+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e0401dbd0 con 0x562c291119b0 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.134+0000 7f9e32005b80 10 client.26287 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.134+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c29193cc0 con 0x562c291119b0 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.134+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e04009ca0 con 0x562c291119b0 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.135+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9e04011070 con 0x562c291119b0 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 10 client.26287 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 20 client.26287 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c291941c0 0x562c291965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 4 client.26287 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:12:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.136+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c29031570 con 0x562c291941c0 2024-05-27T07:12:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.137+0000 7f9e20a00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c291941c0 0x562c291965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.140+0000 7f9e0aa00700 10 client.26287 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:12:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.140+0000 7f9e20a00700 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c291941c0 0x562c291965e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9775d8791601ab0 server_cookie=fd31e403b81dcb02 in_seq=0 out_seq=0 2024-05-27T07:12:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.141+0000 7f9e0aa00700 1 -- 172.21.15.16:0/3298210686 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c29031570 con 0x562c291941c0 2024-05-27T07:12:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.141+0000 7f9e0aa00700 10 client.26287 handle_command_reply: tid=0 2024-05-27T07:12:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.141+0000 7f9e32005b80 1 client.26287 shutdown 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 msgr2=0x562c2912c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 0x562c2912c790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e040b4e50 tx=0x7f9e040ab940 comp rx=0 tx=0).stop 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 shutdown_connections 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c2911a4e0 0x562c29192600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x562c291941c0 0x562c291965e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c28f88e40 0x562c2912c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e0aa00700 0 client.26287 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:12:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 --2- 172.21.15.16:0/3298210686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c291119b0 0x562c2912c790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 >> 172.21.15.16:0/3298210686 conn(0x562c28fe8fc0 msgr2=0x562c29112460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.142+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 shutdown_connections 2024-05-27T07:12:07.134 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:12:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.143+0000 7f9e32005b80 1 -- 172.21.15.16:0/3298210686 wait complete. 2024-05-27T07:12:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.143+0000 7f9e32005b80 20 client.26287 trim_cache size 0 max 16384 2024-05-27T07:12:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 msgr2=0x7f9e000b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 0x7f9e000b5000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c006fd0 tx=0x7f9e0c008040 comp rx=0 tx=0).stop 2024-05-27T07:12:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 msgr2=0x7f9e1c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e04015000 tx=0x7f9e04009800 comp rx=0 tx=0).stop 2024-05-27T07:12:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 shutdown_connections 2024-05-27T07:12:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9e000b2af0 0x7f9e000b5000 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1c109860 0x7f9e1c06e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c1045e0 0x7f9e1c075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 --2- 172.21.15.16:0/951754546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e1c108f10 0x7f9e1c06e3c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.144+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 >> 172.21.15.16:0/951754546 conn(0x7f9e1c100030 msgr2=0x7f9e1c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.145+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 shutdown_connections 2024-05-27T07:12:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:07.145+0000 7f9e23400700 1 -- 172.21.15.16:0/951754546 wait complete. 2024-05-27T07:12:08.279 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:07 smithi016 ceph-mon[31415]: pgmap v349: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 521 op/s 2024-05-27T07:12:08.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:07 smithi164 ceph-mon[35327]: pgmap v349: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 521 op/s 2024-05-27T07:12:08.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:07 smithi069 ceph-mon[35276]: pgmap v349: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 521 op/s 2024-05-27T07:12:08.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:12:09.640 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:12:10.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:09 smithi016 ceph-mon[31415]: pgmap v350: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 415 op/s 2024-05-27T07:12:10.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:09 smithi016 ceph-mon[31415]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:12:10.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:09 smithi164 ceph-mon[35327]: pgmap v350: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 415 op/s 2024-05-27T07:12:10.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:09 smithi164 ceph-mon[35327]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:12:10.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:09 smithi069 ceph-mon[35276]: pgmap v350: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 415 op/s 2024-05-27T07:12:10.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:09 smithi069 ceph-mon[35276]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:12:11.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:10 smithi164 ceph-mon[35327]: pgmap v352: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 442 op/s 2024-05-27T07:12:11.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:10 smithi016 ceph-mon[31415]: pgmap v352: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 442 op/s 2024-05-27T07:12:11.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:10 smithi069 ceph-mon[35276]: pgmap v352: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 442 op/s 2024-05-27T07:12:13.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:13 smithi069 ceph-mon[35276]: pgmap v353: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:12:13.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:13 smithi164 ceph-mon[35327]: pgmap v353: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:12:13.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:13 smithi016 ceph-mon[31415]: pgmap v353: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-05-27T07:12:15.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:15 smithi164 ceph-mon[35327]: pgmap v354: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 390 op/s 2024-05-27T07:12:15.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:15 smithi016 ceph-mon[31415]: pgmap v354: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 390 op/s 2024-05-27T07:12:15.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:15 smithi069 ceph-mon[35276]: pgmap v354: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 390 op/s 2024-05-27T07:12:17.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:17 smithi016 ceph-mon[31415]: pgmap v355: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:12:17.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:17 smithi164 ceph-mon[35327]: pgmap v355: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:12:17.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:17 smithi069 ceph-mon[35276]: pgmap v355: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 317 op/s 2024-05-27T07:12:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 -- 172.21.15.16:0/2420806492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 msgr2=0x7fbb1009ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 --2- 172.21.15.16:0/2420806492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb1009ec00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbb1806a3b0 tx=0x7fbb18069f30 comp rx=0 tx=0).stop 2024-05-27T07:12:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 -- 172.21.15.16:0/2420806492 shutdown_connections 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 --2- 172.21.15.16:0/2420806492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb10099110 0x7fbb100a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 --2- 172.21.15.16:0/2420806492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb1009ec00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 --2- 172.21.15.16:0/2420806492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb100987c0 0x7fbb10098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.063+0000 7fbb1e400700 1 -- 172.21.15.16:0/2420806492 >> 172.21.15.16:0/2420806492 conn(0x7fbb100942c0 msgr2=0x7fbb10096710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.064+0000 7fbb1e400700 1 -- 172.21.15.16:0/2420806492 shutdown_connections 2024-05-27T07:12:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.064+0000 7fbb1e400700 1 -- 172.21.15.16:0/2420806492 wait complete. 2024-05-27T07:12:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.065+0000 7fbb1e400700 1 Processor -- start 2024-05-27T07:12:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.066+0000 7fbb1e400700 1 -- start start 2024-05-27T07:12:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.067+0000 7fbb1e400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100987c0 0x7fbb1012efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.067+0000 7fbb1e400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb10099110 0x7fbb1012f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.067+0000 7fbb1e400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1e400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100a86e0 con 0x7fbb100987c0 2024-05-27T07:12:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1e400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100a8560 con 0x7fbb10099110 2024-05-27T07:12:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1e400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100a8860 con 0x7fbb1009e7f0 2024-05-27T07:12:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42564/0 (socket says 172.21.15.16:42564) 2024-05-27T07:12:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1d000700 1 -- 172.21.15.16:0/1677213974 learned_addr learned my addr 172.21.15.16:0/1677213974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.068+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb10099110 0x7fbb1012f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.069+0000 7fbb1d000700 1 -- 172.21.15.16:0/1677213974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb10099110 msgr2=0x7fbb1012f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.069+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb10099110 0x7fbb1012f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.069+0000 7fbb1d000700 1 -- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100987c0 msgr2=0x7fbb1012efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.069+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100987c0 0x7fbb1012efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.069+0000 7fbb1d000700 1 -- 172.21.15.16:0/1677213974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb10133e80 con 0x7fbb1009e7f0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.071+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c007ee0 tx=0x7fbb0c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.071+0000 7fbb15600700 1 -- 172.21.15.16:0/1677213974 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00dbd0 con 0x7fbb1009e7f0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.071+0000 7fbb15600700 1 -- 172.21.15.16:0/1677213974 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb0c00dd70 con 0x7fbb1009e7f0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.071+0000 7fbb15600700 1 -- 172.21.15.16:0/1677213974 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0058d0 con 0x7fbb1009e7f0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.072+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb10134170 con 0x7fbb1009e7f0 2024-05-27T07:12:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.072+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1013ba90 con 0x7fbb1009e7f0 2024-05-27T07:12:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.085+0000 7fbb15600700 1 -- 172.21.15.16:0/1677213974 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbb0c01a020 con 0x7fbb1009e7f0 2024-05-27T07:12:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.088+0000 7fbb15600700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 0x7fbb000b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.088+0000 7fbb15600700 1 -- 172.21.15.16:0/1677213974 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fbb0c0e2f80 con 0x7fbb1009e7f0 2024-05-27T07:12:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.089+0000 7fbb17e00700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 0x7fbb000b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.184+0000 7fbb17e00700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 0x7fbb000b5000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb180525c0 tx=0x7fbb1806a340 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.187+0000 7fbb2e5f9b80 1 Processor -- start 2024-05-27T07:12:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.187+0000 7fbb2e5f9b80 1 -- start start 2024-05-27T07:12:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.188+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.188+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca821303a0 0x55ca8213ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.188+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.189+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38036/0 (socket says 172.21.15.16:38036) 2024-05-27T07:12:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.189+0000 7fbb17e00700 1 -- 172.21.15.16:0/3153046465 learned_addr learned my addr 172.21.15.16:0/3153046465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.189+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca821303a0 0x55ca8213ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.189+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.192+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca8205ee40 con 0x55ca821307b0 2024-05-27T07:12:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.192+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca8204e9e0 con 0x55ca81fb1e40 2024-05-27T07:12:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.192+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca82052a20 con 0x55ca821303a0 2024-05-27T07:12:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.195+0000 7fbb1d000700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.195+0000 7fbb17e00700 1 -- 172.21.15.16:0/3153046465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca821303a0 msgr2=0x55ca8213ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.195+0000 7fbb17e00700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca821303a0 0x55ca8213ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.195+0000 7fbb17e00700 1 -- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 msgr2=0x55ca8213d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.197+0000 7fbb17e00700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.197+0000 7fbb17e00700 1 -- 172.21.15.16:0/3153046465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca82017420 con 0x55ca81fb1e40 2024-05-27T07:12:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.198+0000 7fbb1d000700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:12:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.200+0000 7fbb17e00700 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb18051ee0 tx=0x7fbb18089000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e1e62b1818142 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.200+0000 7fbb0a000700 1 -- 172.21.15.16:0/3153046465 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1809c070 con 0x55ca81fb1e40 2024-05-27T07:12:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.201+0000 7fbb0a000700 1 -- 172.21.15.16:0/3153046465 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb1807fb10 con 0x55ca81fb1e40 2024-05-27T07:12:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.201+0000 7fbb0a000700 1 -- 172.21.15.16:0/3153046465 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18085040 con 0x55ca81fb1e40 2024-05-27T07:12:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.202+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 msgr2=0x55ca8212fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.202+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb18051ee0 tx=0x7fbb18089000 comp rx=0 tx=0).stop 2024-05-27T07:12:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.205+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 shutdown_connections 2024-05-27T07:12:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.205+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.205+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca81fb1e40 0x55ca8212fe60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.205+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/3153046465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca821303a0 0x55ca8213ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.206+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 >> 172.21.15.16:0/3153046465 conn(0x55ca82011fc0 msgr2=0x55ca8212bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.206+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 shutdown_connections 2024-05-27T07:12:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.207+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/3153046465 wait complete. 2024-05-27T07:12:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.209+0000 7fbb2e5f9b80 1 -- start start 2024-05-27T07:12:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.209+0000 7fbb2e5f9b80 1 Processor -- start 2024-05-27T07:12:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.211+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca8213a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.212+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8213ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.212+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca8213a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.212+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca8213a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42592/0 (socket says 172.21.15.16:42592) 2024-05-27T07:12:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.212+0000 7fbb17e00700 1 -- 172.21.15.16:0/1178117314 learned_addr learned my addr 172.21.15.16:0/1178117314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.213+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca8213b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.213+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca81f54fa0 con 0x55ca821303a0 2024-05-27T07:12:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.213+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8213ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.215+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca8213b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.215+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca81fef760 con 0x55ca821307b0 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.215+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ca8213d970 con 0x55ca81fb1e40 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.215+0000 7fbb2e5f9b80 10 client.0 _ll_register_callbacks cb 0x55ca81ff8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 -- 172.21.15.16:0/1178117314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 msgr2=0x55ca8213a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca8213a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 -- 172.21.15.16:0/1178117314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 msgr2=0x55ca8213ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8213ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca81f9e8d0 con 0x55ca821307b0 2024-05-27T07:12:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8213ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbb1d000700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca8213b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0e0230 tx=0x7fbb0c0195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c960543600f358a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.216+0000 7fbafea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.217+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0a9dc0 con 0x55ca821307b0 2024-05-27T07:12:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.217+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1807fb10 con 0x55ca821307b0 2024-05-27T07:12:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.217+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fbb0c0c4440 con 0x55ca821307b0 2024-05-27T07:12:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.217+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c0c9050 con 0x55ca821307b0 2024-05-27T07:12:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.217+0000 7fbb2e5f9b80 10 client.36359 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.218+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ca8201f060 con 0x55ca821307b0 2024-05-27T07:12:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.218+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb1807fb10 con 0x55ca821307b0 2024-05-27T07:12:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.218+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbb0c0b2790 con 0x55ca821307b0 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 10 client.36359 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 10 client.36359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 20 client.36359 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca82153170 0x55ca82155590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 4 client.36359 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca82060d90 con 0x55ca82153170 2024-05-27T07:12:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.219+0000 7fbb17e00700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca82153170 0x55ca82155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.221+0000 7fbb17e00700 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca82153170 0x55ca82155590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1072f4079474f8 server_cookie=487ba07636da150e in_seq=0 out_seq=0 2024-05-27T07:12:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.221+0000 7fbafea00700 10 client.36359 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:12:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.223+0000 7fbafea00700 1 -- 172.21.15.16:0/1178117314 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ca82060d90 con 0x55ca82153170 2024-05-27T07:12:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.223+0000 7fbafea00700 10 client.36359 handle_command_reply: tid=0 2024-05-27T07:12:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.223+0000 7fbb2e5f9b80 1 client.36359 shutdown 2024-05-27T07:12:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 msgr2=0x55ca8213b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca8213b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0e0230 tx=0x7fbb0c0195c0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 shutdown_connections 2024-05-27T07:12:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca82153170 0x55ca82155590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca8213b0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca8213a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/1178117314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8213ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 >> 172.21.15.16:0/1178117314 conn(0x55ca82011fc0 msgr2=0x55ca8212aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbafea00700 0 client.36359 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 shutdown_connections 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.224+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/1178117314 wait complete. 2024-05-27T07:12:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.225+0000 7fbb2e5f9b80 20 client.36359 trim_cache size 0 max 16384 2024-05-27T07:12:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.262+0000 7fbb2e5f9b80 1 Processor -- start 2024-05-27T07:12:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.262+0000 7fbb2e5f9b80 1 -- start start 2024-05-27T07:12:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca821398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 0x55ca82135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10140660 con 0x55ca821307b0 2024-05-27T07:12:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101421a0 con 0x55ca821303a0 2024-05-27T07:12:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.263+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10140d30 con 0x55ca81fb1e40 2024-05-27T07:12:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.265+0000 7fbb1fa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 0x55ca82135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.265+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca821398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.265+0000 7fbb1d000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.265+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca821398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:42608/0 (socket says 172.21.15.16:42608) 2024-05-27T07:12:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.265+0000 7fbb17e00700 1 -- 172.21.15.16:0/2399397409 learned_addr learned my addr 172.21.15.16:0/2399397409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.266+0000 7fbb1fa00700 1 -- 172.21.15.16:0/2399397409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 msgr2=0x55ca821398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.266+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca821398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.266+0000 7fbb1fa00700 1 -- 172.21.15.16:0/2399397409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 msgr2=0x55ca8213b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.266+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.266+0000 7fbb1fa00700 1 -- 172.21.15.16:0/2399397409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca82123830 con 0x55ca821303a0 2024-05-27T07:12:18.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.269+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 0x55ca82135e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ca8212d440 tx=0x7fbb04010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34890279cbdfde74 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.271+0000 7fbb09600700 1 -- 172.21.15.16:0/2399397409 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04021070 con 0x55ca821303a0 2024-05-27T07:12:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.271+0000 7fbb09600700 1 -- 172.21.15.16:0/2399397409 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb040044e0 con 0x55ca821303a0 2024-05-27T07:12:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.271+0000 7fbb09600700 1 -- 172.21.15.16:0/2399397409 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04003df0 con 0x55ca821303a0 2024-05-27T07:12:18.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.272+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2399397409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 msgr2=0x55ca82135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.272+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 0x55ca82135e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ca8212d440 tx=0x7fbb04010710 comp rx=0 tx=0).stop 2024-05-27T07:12:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.272+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2399397409 shutdown_connections 2024-05-27T07:12:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.272+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821307b0 0x55ca8213b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.273+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca821398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.273+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2399397409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821303a0 0x55ca82135e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.273+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2399397409 >> 172.21.15.16:0/2399397409 conn(0x55ca82011fc0 msgr2=0x55ca8213fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.277+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2399397409 shutdown_connections 2024-05-27T07:12:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.277+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2399397409 wait complete. 2024-05-27T07:12:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.278+0000 7fbb2e5f9b80 1 -- start start 2024-05-27T07:12:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.278+0000 7fbb2e5f9b80 1 Processor -- start 2024-05-27T07:12:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.280+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca82149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.280+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8214a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.280+0000 7fbb2e5f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10141ad0 con 0x55ca821303a0 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10141400 con 0x55ca821307b0 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb2e5f9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1013e5c0 con 0x55ca81fb1e40 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb2e5f9b80 10 client.0 _ll_register_callbacks cb 0x55ca81ff8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb1d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb1d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38066/0 (socket says 172.21.15.16:38066) 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.281+0000 7fbb1d000700 1 -- 172.21.15.16:0/2221694141 learned_addr learned my addr 172.21.15.16:0/2221694141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb17e00700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca82149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1d000700 1 -- 172.21.15.16:0/2221694141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 msgr2=0x55ca82149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1fa00700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8214a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1d000700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca82149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1d000700 1 -- 172.21.15.16:0/2221694141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 msgr2=0x55ca8214a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1d000700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8214a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.282+0000 7fbb1d000700 1 -- 172.21.15.16:0/2221694141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca821bc0e0 con 0x55ca821307b0 2024-05-27T07:12:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.283+0000 7fbb17e00700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca82149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:12:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.283+0000 7fbb1d000700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0a9a70 tx=0x7fbb0c0aa630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b279303f1abaf378 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.283+0000 7fbafea00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.284+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c088030 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.285+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fbb0c0a63b0 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.285+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb040044e0 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.285+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c0a9dc0 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 10 client.36365 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ca821bd310 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb040044e0 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbb0c0a5730 con 0x55ca821307b0 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 10 client.36365 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 10 client.36365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 20 client.36365 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.286+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca821bda20 0x55ca821bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.287+0000 7fbb2e5f9b80 4 client.36365 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:12:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.287+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca8205a570 con 0x55ca821bda20 2024-05-27T07:12:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.288+0000 7fbb17e00700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca821bda20 0x55ca821bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.289+0000 7fbafea00700 10 client.36365 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:12:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.290+0000 7fbb17e00700 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca821bda20 0x55ca821bfe40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=249e1de57f01bdb3 server_cookie=ed2860abb7277979 in_seq=0 out_seq=0 2024-05-27T07:12:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.290+0000 7fbafea00700 1 -- 172.21.15.16:0/2221694141 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca8205a570 con 0x55ca821bda20 2024-05-27T07:12:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.290+0000 7fbafea00700 10 client.36365 handle_command_reply: tid=0 2024-05-27T07:12:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 client.36365 shutdown 2024-05-27T07:12:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 msgr2=0x55ca821bbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0a9a70 tx=0x7fbb0c0aa630 comp rx=0 tx=0).stop 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 shutdown_connections 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x55ca821bda20 0x55ca821bfe40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca821307b0 0x55ca821bbba0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca81fb1e40 0x55ca82149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 --2- 172.21.15.16:0/2221694141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca821303a0 0x55ca8214a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 >> 172.21.15.16:0/2221694141 conn(0x55ca82011fc0 msgr2=0x55ca8213ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.292+0000 7fbafea00700 0 client.36365 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.293+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 shutdown_connections 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.293+0000 7fbb2e5f9b80 1 -- 172.21.15.16:0/2221694141 wait complete. 2024-05-27T07:12:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.293+0000 7fbb2e5f9b80 20 client.36365 trim_cache size 0 max 16384 2024-05-27T07:12:18.288 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:12:18.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 msgr2=0x7fbb000b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 0x7fbb000b5000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb180525c0 tx=0x7fbb1806a340 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 msgr2=0x7fbb101337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c007ee0 tx=0x7fbb0c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 shutdown_connections 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbb000b2af0 0x7fbb000b5000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb1009e7f0 0x7fbb101337e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100987c0 0x7fbb1012efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 --2- 172.21.15.16:0/1677213974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb10099110 0x7fbb1012f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.294+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 >> 172.21.15.16:0/1677213974 conn(0x7fbb100942c0 msgr2=0x7fbb10095770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.295+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 shutdown_connections 2024-05-27T07:12:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:18.295+0000 7fbb1e400700 1 -- 172.21.15.16:0/1677213974 wait complete. 2024-05-27T07:12:19.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:19 smithi016 ceph-mon[31415]: pgmap v356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 326 op/s 2024-05-27T07:12:19.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:12:19.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:19 smithi164 ceph-mon[35327]: pgmap v356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 326 op/s 2024-05-27T07:12:19.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:19 smithi069 ceph-mon[35276]: pgmap v356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 326 op/s 2024-05-27T07:12:20.211 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:12:21.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:21 smithi016 ceph-mon[31415]: pgmap v357: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 308 op/s 2024-05-27T07:12:21.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:21 smithi164 ceph-mon[35327]: pgmap v357: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 308 op/s 2024-05-27T07:12:21.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:21 smithi069 ceph-mon[35276]: pgmap v357: 129 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 92 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 308 op/s 2024-05-27T07:12:23.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:23 smithi164 ceph-mon[35327]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 19 MiB/s wr, 392 op/s 2024-05-27T07:12:23.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:23 smithi016 ceph-mon[31415]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 19 MiB/s wr, 392 op/s 2024-05-27T07:12:23.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:23 smithi069 ceph-mon[35276]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 19 MiB/s wr, 392 op/s 2024-05-27T07:12:25.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:25 smithi164 ceph-mon[35327]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 268 op/s 2024-05-27T07:12:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:25 smithi016 ceph-mon[31415]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 268 op/s 2024-05-27T07:12:25.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:25 smithi069 ceph-mon[35276]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 104 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 268 op/s 2024-05-27T07:12:27.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:27 smithi164 ceph-mon[35327]: pgmap v360: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 377 op/s 2024-05-27T07:12:27.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:27 smithi016 ceph-mon[31415]: pgmap v360: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 377 op/s 2024-05-27T07:12:27.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:27 smithi069 ceph-mon[35276]: pgmap v360: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 377 op/s 2024-05-27T07:12:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:29 smithi164 ceph-mon[35327]: pgmap v361: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 18 MiB/s wr, 373 op/s 2024-05-27T07:12:30.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:29 smithi164 ceph-mon[35327]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:12:30.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:29 smithi016 ceph-mon[31415]: pgmap v361: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 18 MiB/s wr, 373 op/s 2024-05-27T07:12:30.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:29 smithi016 ceph-mon[31415]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:12:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:29 smithi069 ceph-mon[35276]: pgmap v361: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 18 MiB/s wr, 373 op/s 2024-05-27T07:12:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:29 smithi069 ceph-mon[35276]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.436+0000 7f0ee3a00700 1 -- 172.21.15.16:0/2086227488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc10e8d0 msgr2=0x7f0edc110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.436+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/2086227488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc10e8d0 0x7f0edc110db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80098f0 tx=0x7f0ec8009c00 comp rx=0 tx=0).stop 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.437+0000 7f0ee3a00700 1 -- 172.21.15.16:0/2086227488 shutdown_connections 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.437+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/2086227488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc10e8d0 0x7f0edc110db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.437+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/2086227488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc1054f0 0x7f0edc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.437+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/2086227488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc105e40 0x7f0edc10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.437+0000 7f0ee3a00700 1 -- 172.21.15.16:0/2086227488 >> 172.21.15.16:0/2086227488 conn(0x7f0edc0fedb0 msgr2=0x7f0edc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.439+0000 7f0ee3a00700 1 -- 172.21.15.16:0/2086227488 shutdown_connections 2024-05-27T07:12:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.439+0000 7f0ee3a00700 1 -- 172.21.15.16:0/2086227488 wait complete. 2024-05-27T07:12:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.442+0000 7f0ee3a00700 1 Processor -- start 2024-05-27T07:12:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.442+0000 7f0ee3a00700 1 -- start start 2024-05-27T07:12:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.444+0000 7f0ee3a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 0x7f0edc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee3a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 0x7f0edc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee3a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc10e8d0 0x7f0edc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee3a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1121d0 con 0x7f0edc1054f0 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee3a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc112050 con 0x7f0edc105e40 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee3a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc112350 con 0x7f0edc10e8d0 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee1000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 0x7f0edc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee1000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 0x7f0edc19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45006/0 (socket says 172.21.15.16:45006) 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0ee1000700 1 -- 172.21.15.16:0/3621348058 learned_addr learned my addr 172.21.15.16:0/3621348058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.446+0000 7f0edbe00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 0x7f0edc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0ee1a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc10e8d0 0x7f0edc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0edbe00700 1 -- 172.21.15.16:0/3621348058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc10e8d0 msgr2=0x7f0edc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0edbe00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc10e8d0 0x7f0edc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0edbe00700 1 -- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 msgr2=0x7f0edc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0edbe00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 0x7f0edc19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.447+0000 7f0edbe00700 1 -- 172.21.15.16:0/3621348058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc198fc0 con 0x7f0edc105e40 2024-05-27T07:12:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0edbe00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 0x7f0edc19ec30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400ef10 tx=0x7f0ed400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0ed9600700 1 -- 172.21.15.16:0/3621348058 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400c9c0 con 0x7f0edc105e40 2024-05-27T07:12:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0ed9600700 1 -- 172.21.15.16:0/3621348058 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ed4019070 con 0x7f0edc105e40 2024-05-27T07:12:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc199250 con 0x7f0edc105e40 2024-05-27T07:12:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0ed9600700 1 -- 172.21.15.16:0/3621348058 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4014480 con 0x7f0edc105e40 2024-05-27T07:12:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.449+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc1a5580 con 0x7f0edc105e40 2024-05-27T07:12:30.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.461+0000 7f0ed9600700 1 -- 172.21.15.16:0/3621348058 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0ed4012050 con 0x7f0edc105e40 2024-05-27T07:12:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.463+0000 7f0ed9600700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 0x7f0ec00b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.464+0000 7f0ed9600700 1 -- 172.21.15.16:0/3621348058 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0ed40e3770 con 0x7f0edc105e40 2024-05-27T07:12:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.465+0000 7f0ee1000700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 0x7f0ec00b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.480+0000 7f0ee1000700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 0x7f0ec00b9410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0edc199a50 tx=0x7f0ecc00b500 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.482+0000 7f0ef24e1b80 1 Processor -- start 2024-05-27T07:12:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 -- start start 2024-05-27T07:12:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756189f0 0x55bb756ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756abe70 0x55bb756b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb755d9e40 con 0x55bb756abe70 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb755c99e0 con 0x55bb7552ce40 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.483+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb755cda20 con 0x55bb756189f0 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51636/0 (socket says 172.21.15.16:51636) 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 -- 172.21.15.16:0/2996884615 learned_addr learned my addr 172.21.15.16:0/2996884615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 -- 172.21.15.16:0/2996884615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756189f0 msgr2=0x55bb756ab330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756189f0 0x55bb756ab330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 -- 172.21.15.16:0/2996884615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756abe70 msgr2=0x55bb756b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756abe70 0x55bb756b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.484+0000 7f0edbe00700 1 -- 172.21.15.16:0/2996884615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb75592420 con 0x55bb7552ce40 2024-05-27T07:12:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.485+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40090a0 tx=0x7f0ed40aa4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c2fd0c45dbfc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.485+0000 7f0ed2000700 1 -- 172.21.15.16:0/2996884615 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40aad00 con 0x55bb7552ce40 2024-05-27T07:12:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.485+0000 7f0ed2000700 1 -- 172.21.15.16:0/2996884615 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ed40aa600 con 0x55bb7552ce40 2024-05-27T07:12:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.485+0000 7f0ed2000700 1 -- 172.21.15.16:0/2996884615 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40aad00 con 0x55bb7552ce40 2024-05-27T07:12:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.487+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2996884615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 msgr2=0x55bb756ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.487+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40090a0 tx=0x7f0ed40aa4f0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2996884615 shutdown_connections 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756abe70 0x55bb756b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756189f0 0x55bb756ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2996884615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756ab870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2996884615 >> 172.21.15.16:0/2996884615 conn(0x55bb755c6390 msgr2=0x55bb756a74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.488+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2996884615 shutdown_connections 2024-05-27T07:12:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.489+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2996884615 wait complete. 2024-05-27T07:12:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.494+0000 7f0ef24e1b80 1 -- start start 2024-05-27T07:12:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.495+0000 7f0ef24e1b80 1 Processor -- start 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb754cffa0 con 0x55bb756189f0 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb7556a760 con 0x55bb7552ce40 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.496+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb756b8b20 con 0x55bb756abe70 2024-05-27T07:12:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.497+0000 7f0ef24e1b80 10 client.0 _ll_register_callbacks cb 0x55bb75573c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.497+0000 7f0ee1000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51640/0 (socket says 172.21.15.16:51640) 2024-05-27T07:12:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1000700 1 -- 172.21.15.16:0/1682286082 learned_addr learned my addr 172.21.15.16:0/1682286082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 -- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 msgr2=0x55bb756b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 -- 172.21.15.16:0/1682286082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 msgr2=0x55bb756b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.498+0000 7f0ee1a00700 1 -- 172.21.15.16:0/1682286082 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb755198d0 con 0x55bb756abe70 2024-05-27T07:12:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.499+0000 7f0ec6a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:12:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.499+0000 7f0ee1a00700 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756b6200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c010 tx=0x7f0ec800f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d6754cc6065e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.499+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800f950 con 0x55bb756abe70 2024-05-27T07:12:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.500+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0ec8005240 con 0x55bb756abe70 2024-05-27T07:12:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.500+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed40aa600 con 0x55bb756abe70 2024-05-27T07:12:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.500+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ec8005440 con 0x55bb756abe70 2024-05-27T07:12:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.500+0000 7f0ef24e1b80 10 client.26311 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.500+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb7559a060 con 0x55bb756abe70 2024-05-27T07:12:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.501+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ed40aa600 con 0x55bb756abe70 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.501+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ec8029030 con 0x55bb756abe70 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 10 client.26311 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 10 client.26311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 20 client.26311 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756d6da0 0x55bb756d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 4 client.26311 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.502+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb755dbd90 con 0x55bb756d6da0 2024-05-27T07:12:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.505+0000 7f0ee1000700 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756d6da0 0x55bb756d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.506+0000 7f0ee1000700 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756d6da0 0x55bb756d91e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dd932a7c5b4fcd5 server_cookie=658bd695693bf679 in_seq=0 out_seq=0 2024-05-27T07:12:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.506+0000 7f0ec6a00700 10 client.26311 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:12:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ec6a00700 1 -- 172.21.15.16:0/1682286082 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bb755dbd90 con 0x55bb756d6da0 2024-05-27T07:12:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ec6a00700 10 client.26311 handle_command_reply: tid=0 2024-05-27T07:12:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 client.26311 shutdown 2024-05-27T07:12:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 msgr2=0x55bb756b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756b6200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c010 tx=0x7f0ec800f460 comp rx=0 tx=0).stop 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 shutdown_connections 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756d6da0 0x55bb756d91e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756b6200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.508+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/1682286082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.509+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 >> 172.21.15.16:0/1682286082 conn(0x55bb755c6390 msgr2=0x55bb756ab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.509+0000 7f0ec6a00700 0 client.26311 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:12:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.509+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 shutdown_connections 2024-05-27T07:12:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.510+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/1682286082 wait complete. 2024-05-27T07:12:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.510+0000 7f0ef24e1b80 20 client.26311 trim_cache size 0 max 16384 2024-05-27T07:12:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.544+0000 7f0ef24e1b80 1 Processor -- start 2024-05-27T07:12:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 -- start start 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1a5e00 con 0x55bb756189f0 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1a7940 con 0x55bb7552ce40 2024-05-27T07:12:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.547+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1a64d0 con 0x55bb756abe70 2024-05-27T07:12:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.549+0000 7f0ee1000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.549+0000 7f0ee1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.549+0000 7f0ee1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55112/0 (socket says 172.21.15.16:55112) 2024-05-27T07:12:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.549+0000 7f0ee1a00700 1 -- 172.21.15.16:0/3226850684 learned_addr learned my addr 172.21.15.16:0/3226850684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1000700 1 -- 172.21.15.16:0/3226850684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 msgr2=0x55bb756a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1000700 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1000700 1 -- 172.21.15.16:0/3226850684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 msgr2=0x55bb756a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1000700 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1000700 1 -- 172.21.15.16:0/3226850684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb756a6b10 con 0x55bb7552ce40 2024-05-27T07:12:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.550+0000 7f0ee1a00700 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:12:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.551+0000 7f0ee1000700 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b0f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55bb756d9950 tx=0x7f0ecc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8046069d69a28f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.553+0000 7f0ed1600700 1 -- 172.21.15.16:0/3226850684 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc003880 con 0x55bb7552ce40 2024-05-27T07:12:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.553+0000 7f0ed1600700 1 -- 172.21.15.16:0/3226850684 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ecc004540 con 0x55bb7552ce40 2024-05-27T07:12:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.553+0000 7f0ed1600700 1 -- 172.21.15.16:0/3226850684 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc027b90 con 0x55bb7552ce40 2024-05-27T07:12:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.555+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/3226850684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 msgr2=0x55bb756b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.555+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b0f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55bb756d9950 tx=0x7f0ecc013000 comp rx=0 tx=0).stop 2024-05-27T07:12:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/3226850684 shutdown_connections 2024-05-27T07:12:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7552ce40 0x55bb756b0f30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/3226850684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb756189f0 0x55bb756a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/3226850684 >> 172.21.15.16:0/3226850684 conn(0x55bb755c6390 msgr2=0x55bb756ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.556+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/3226850684 shutdown_connections 2024-05-27T07:12:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.559+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/3226850684 wait complete. 2024-05-27T07:12:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.559+0000 7f0ef24e1b80 1 -- start start 2024-05-27T07:12:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.560+0000 7f0ef24e1b80 1 Processor -- start 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 0x55bb756b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1a7270 con 0x55bb7552ce40 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1a6ba0 con 0x55bb756189f0 2024-05-27T07:12:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc073fa0 con 0x55bb756abe70 2024-05-27T07:12:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.561+0000 7f0ef24e1b80 10 client.0 _ll_register_callbacks cb 0x55bb75573c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:30.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0edbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0ee1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0ee1a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:55124/0 (socket says 172.21.15.16:55124) 2024-05-27T07:12:30.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0edbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:51658/0 (socket says 172.21.15.16:51658) 2024-05-27T07:12:30.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0edbe00700 1 -- 172.21.15.16:0/2679786233 learned_addr learned my addr 172.21.15.16:0/2679786233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:30.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.563+0000 7f0ee1000700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 0x55bb756b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.565+0000 7f0edbe00700 1 -- 172.21.15.16:0/2679786233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 msgr2=0x55bb756af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.565+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.565+0000 7f0edbe00700 1 -- 172.21.15.16:0/2679786233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 msgr2=0x55bb756b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.566+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 0x55bb756b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.566+0000 7f0edbe00700 1 -- 172.21.15.16:0/2679786233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb756afe50 con 0x55bb756189f0 2024-05-27T07:12:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.566+0000 7f0ee1000700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 0x55bb756b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.567+0000 7f0ec6a00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.567+0000 7f0edbe00700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40aa300 tx=0x7f0ed40aaa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e34229a88e6056 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.568+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40aa600 con 0x55bb756189f0 2024-05-27T07:12:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.568+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ecc004540 con 0x55bb756189f0 2024-05-27T07:12:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.568+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0ed40a6bf0 con 0x55bb756189f0 2024-05-27T07:12:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.570+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ed40ae9b0 con 0x55bb756189f0 2024-05-27T07:12:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.570+0000 7f0ef24e1b80 10 client.36383 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.570+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb756bdd00 con 0x55bb756189f0 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.570+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ecc004540 con 0x55bb756189f0 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ed400cb60 con 0x55bb756189f0 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 10 client.36383 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 10 client.36383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 20 client.36383 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756bded0 0x55bb756be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 4 client.36383 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:12:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.571+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb755d5570 con 0x55bb756bded0 2024-05-27T07:12:30.566 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:12:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.572+0000 7f0ee1000700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756bded0 0x55bb756be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:30.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.573+0000 7f0ee1000700 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756bded0 0x55bb756be2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15778a59e05a63b0 server_cookie=cabdc864386a795b in_seq=0 out_seq=0 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.573+0000 7f0ec6a00700 10 client.36383 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.573+0000 7f0ec6a00700 1 -- 172.21.15.16:0/2679786233 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb755d5570 con 0x55bb756bded0 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.573+0000 7f0ec6a00700 10 client.36383 handle_command_reply: tid=0 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.573+0000 7f0ef24e1b80 1 client.36383 shutdown 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 msgr2=0x55bb756af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40aa300 tx=0x7f0ed40aaa90 comp rx=0 tx=0).stop 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 shutdown_connections 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55bb756bded0 0x55bb756be2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb756abe70 0x55bb756af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7552ce40 0x55bb756b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 --2- 172.21.15.16:0/2679786233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb756189f0 0x55bb756af150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 >> 172.21.15.16:0/2679786233 conn(0x55bb755c6390 msgr2=0x55bb756ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.574+0000 7f0ec6a00700 0 client.36383 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.575+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 shutdown_connections 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.575+0000 7f0ef24e1b80 1 -- 172.21.15.16:0/2679786233 wait complete. 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.575+0000 7f0ef24e1b80 20 client.36383 trim_cache size 0 max 16384 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.577+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 msgr2=0x7f0ec00b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.577+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 0x7f0ec00b9410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0edc199a50 tx=0x7f0ecc00b500 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.577+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 msgr2=0x7f0edc19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.577+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 0x7f0edc19ec30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400ef10 tx=0x7f0ed400c3a0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 shutdown_connections 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f0ec00b6f00 0x7f0ec00b9410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0edc10e8d0 0x7f0edc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0edc1054f0 0x7f0edc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 --2- 172.21.15.16:0/3621348058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0edc105e40 0x7f0edc19ec30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 >> 172.21.15.16:0/3621348058 conn(0x7f0edc0fedb0 msgr2=0x7f0edc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 shutdown_connections 2024-05-27T07:12:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:30.579+0000 7f0ee3a00700 1 -- 172.21.15.16:0/3621348058 wait complete. 2024-05-27T07:12:32.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:31 smithi164 ceph-mon[35327]: pgmap v363: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 393 op/s 2024-05-27T07:12:32.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:31 smithi016 ceph-mon[31415]: pgmap v363: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 393 op/s 2024-05-27T07:12:32.120 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:31 smithi069 ceph-mon[35276]: pgmap v363: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 393 op/s 2024-05-27T07:12:33.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:12:34.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:33 smithi164 ceph-mon[35327]: pgmap v364: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 20 MiB/s wr, 409 op/s 2024-05-27T07:12:34.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:33 smithi016 ceph-mon[31415]: pgmap v364: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 20 MiB/s wr, 409 op/s 2024-05-27T07:12:34.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:33 smithi069 ceph-mon[35276]: pgmap v364: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 20 MiB/s wr, 409 op/s 2024-05-27T07:12:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:12:36.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:35 smithi164 ceph-mon[35327]: pgmap v365: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 437 op/s 2024-05-27T07:12:36.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:35 smithi016 ceph-mon[31415]: pgmap v365: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 437 op/s 2024-05-27T07:12:36.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:35 smithi069 ceph-mon[35276]: pgmap v365: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 437 op/s 2024-05-27T07:12:38.082 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:37 smithi164 ceph-mon[35327]: pgmap v366: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:38.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:37 smithi016 ceph-mon[31415]: pgmap v366: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:38.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:37 smithi069 ceph-mon[35276]: pgmap v366: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:39 smithi164 ceph-mon[35327]: pgmap v367: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 446 op/s 2024-05-27T07:12:40.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:39 smithi164 ceph-mon[35327]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:12:40.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:39 smithi016 ceph-mon[31415]: pgmap v367: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 446 op/s 2024-05-27T07:12:40.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:39 smithi016 ceph-mon[31415]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:12:40.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:39 smithi069 ceph-mon[35276]: pgmap v367: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 446 op/s 2024-05-27T07:12:40.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:39 smithi069 ceph-mon[35276]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:12:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.798+0000 7ff48da00700 1 -- 172.21.15.16:0/96701635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 msgr2=0x7ff488109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.798+0000 7ff48da00700 1 --2- 172.21.15.16:0/96701635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff488109320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff474009990 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.800+0000 7ff48da00700 1 -- 172.21.15.16:0/96701635 shutdown_connections 2024-05-27T07:12:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.800+0000 7ff48da00700 1 --2- 172.21.15.16:0/96701635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488109860 0x7ff4881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.800+0000 7ff48da00700 1 --2- 172.21.15.16:0/96701635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.800+0000 7ff48da00700 1 --2- 172.21.15.16:0/96701635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff488109320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.800+0000 7ff48da00700 1 -- 172.21.15.16:0/96701635 >> 172.21.15.16:0/96701635 conn(0x7ff488100030 msgr2=0x7ff488102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.801+0000 7ff48da00700 1 -- 172.21.15.16:0/96701635 shutdown_connections 2024-05-27T07:12:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.801+0000 7ff48da00700 1 -- 172.21.15.16:0/96701635 wait complete. 2024-05-27T07:12:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.803+0000 7ff48da00700 1 Processor -- start 2024-05-27T07:12:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.804+0000 7ff48da00700 1 -- start start 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff4881a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488109860 0x7ff4881ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881143c0 con 0x7ff488109860 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff488114240 con 0x7ff488108f10 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff48da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff488114540 con 0x7ff4881045e0 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff486000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff4881a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.805+0000 7ff486000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff4881a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53208/0 (socket says 172.21.15.16:53208) 2024-05-27T07:12:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.806+0000 7ff486000700 1 -- 172.21.15.16:0/2198208548 learned_addr learned my addr 172.21.15.16:0/2198208548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.806+0000 7ff486a00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.807+0000 7ff486a00700 1 -- 172.21.15.16:0/2198208548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 msgr2=0x7ff4881a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.807+0000 7ff486a00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff4881a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.807+0000 7ff486a00700 1 -- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488109860 msgr2=0x7ff4881ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.807+0000 7ff486a00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488109860 0x7ff4881ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.807+0000 7ff486a00700 1 -- 172.21.15.16:0/2198208548 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4881ad660 con 0x7ff4881045e0 2024-05-27T07:12:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.809+0000 7ff486a00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881a2270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c8f0 tx=0x7ff47800cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.810+0000 7ff47f400700 1 -- 172.21.15.16:0/2198208548 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478007940 con 0x7ff4881045e0 2024-05-27T07:12:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.810+0000 7ff47f400700 1 -- 172.21.15.16:0/2198208548 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47800f450 con 0x7ff4881045e0 2024-05-27T07:12:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.810+0000 7ff47f400700 1 -- 172.21.15.16:0/2198208548 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478018660 con 0x7ff4881045e0 2024-05-27T07:12:40.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.811+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4881ad8f0 con 0x7ff4881045e0 2024-05-27T07:12:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.813+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4881addb0 con 0x7ff4881045e0 2024-05-27T07:12:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.819+0000 7ff47f400700 1 -- 172.21.15.16:0/2198208548 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff47800f900 con 0x7ff4881045e0 2024-05-27T07:12:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.822+0000 7ff47f400700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 0x7ff46c0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.822+0000 7ff47f400700 1 -- 172.21.15.16:0/2198208548 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff4780e1f20 con 0x7ff4881045e0 2024-05-27T07:12:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.824+0000 7ff486000700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 0x7ff46c0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.826+0000 7ff486000700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 0x7ff46c0b5110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4740037c0 tx=0x7ff47400b540 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.841+0000 7ff49c4e0b80 1 Processor -- start 2024-05-27T07:12:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.841+0000 7ff49c4e0b80 1 -- start start 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab3b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56404aa69e40 con 0x56404a9bce40 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56404aa599e0 con 0x56404ab41d30 2024-05-27T07:12:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56404aa5da20 con 0x56404ab3bdb0 2024-05-27T07:12:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff486a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff486a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55846/0 (socket says 172.21.15.16:55846) 2024-05-27T07:12:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.844+0000 7ff486a00700 1 -- 172.21.15.16:0/3951628262 learned_addr learned my addr 172.21.15.16:0/3951628262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.845+0000 7ff487400700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.845+0000 7ff486000700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab3b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.845+0000 7ff486a00700 1 -- 172.21.15.16:0/3951628262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 msgr2=0x56404ab3c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.846+0000 7ff486a00700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab3c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.846+0000 7ff486a00700 1 -- 172.21.15.16:0/3951628262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 msgr2=0x56404ab3b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.846+0000 7ff486a00700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab3b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.846+0000 7ff486a00700 1 -- 172.21.15.16:0/3951628262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56404aa22420 con 0x56404a9bce40 2024-05-27T07:12:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.847+0000 7ff486a00700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff4780a4ee0 tx=0x7ff4780a8a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede471feaa61f538 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.848+0000 7ff486000700 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab3b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.848+0000 7ff473e00700 1 -- 172.21.15.16:0/3951628262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4780ad210 con 0x56404a9bce40 2024-05-27T07:12:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.848+0000 7ff473e00700 1 -- 172.21.15.16:0/3951628262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4780b1cc0 con 0x56404a9bce40 2024-05-27T07:12:40.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.848+0000 7ff473e00700 1 -- 172.21.15.16:0/3951628262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4780b1a90 con 0x56404a9bce40 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3951628262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 msgr2=0x56404ab417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff4780a4ee0 tx=0x7ff4780a8a70 comp rx=0 tx=0).stop 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3951628262 shutdown_connections 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab3c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab417f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3951628262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab3b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.851+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3951628262 >> 172.21.15.16:0/3951628262 conn(0x56404aa1cfc0 msgr2=0x56404ab368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.852+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3951628262 shutdown_connections 2024-05-27T07:12:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.852+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3951628262 wait complete. 2024-05-27T07:12:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.853+0000 7ff49c4e0b80 1 -- start start 2024-05-27T07:12:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.853+0000 7ff49c4e0b80 1 Processor -- start 2024-05-27T07:12:40.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 0x56404ab45510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404ab3bdb0 0x56404ab45a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56404a95ffa0 con 0x56404ab3bdb0 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56404a9fa760 con 0x56404a9bce40 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56404ab48800 con 0x56404ab41d30 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.854+0000 7ff49c4e0b80 10 client.0 _ll_register_callbacks cb 0x56404aa03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.855+0000 7ff487400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.855+0000 7ff487400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab45f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:34896/0 (socket says 172.21.15.16:34896) 2024-05-27T07:12:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.855+0000 7ff487400700 1 -- 172.21.15.16:0/3799820591 learned_addr learned my addr 172.21.15.16:0/3799820591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.856+0000 7ff486a00700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 0x56404ab45510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 -- 172.21.15.16:0/3799820591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 msgr2=0x56404ab45f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab45f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 -- 172.21.15.16:0/3799820591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404ab3bdb0 msgr2=0x56404ab45a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404ab3bdb0 0x56404ab45a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 -- 172.21.15.16:0/3799820591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56404a9a98d0 con 0x56404a9bce40 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff486a00700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 0x56404ab45510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c8c0 tx=0x7ff4780a92f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47be6dd7af4256fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.857+0000 7ff470c00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.858+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4780c7040 con 0x56404a9bce40 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.858+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff4780a5450 con 0x56404a9bce40 2024-05-27T07:12:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.858+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4780b1cc0 con 0x56404a9bce40 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4780c7040 con 0x56404a9bce40 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff49c4e0b80 10 client.36395 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56404aa2a060 con 0x56404a9bce40 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4780b1cc0 con 0x56404a9bce40 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff4780a47d0 con 0x56404a9bce40 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff49c4e0b80 10 client.36395 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff49c4e0b80 10 client.36395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.859+0000 7ff49c4e0b80 20 client.36395 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.860+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404ab55600 0x56404ab57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.860+0000 7ff49c4e0b80 4 client.36395 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:12:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.860+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56404aa6bd90 con 0x56404ab55600 2024-05-27T07:12:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.860+0000 7ff486000700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404ab55600 0x56404ab57a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.862+0000 7ff486000700 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404ab55600 0x56404ab57a20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c426eaa91d57a98b server_cookie=3500aaf729ae8a10 in_seq=0 out_seq=0 2024-05-27T07:12:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.862+0000 7ff470c00700 10 client.36395 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.863+0000 7ff470c00700 1 -- 172.21.15.16:0/3799820591 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56404aa6bd90 con 0x56404ab55600 2024-05-27T07:12:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.863+0000 7ff470c00700 10 client.36395 handle_command_reply: tid=0 2024-05-27T07:12:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.867+0000 7ff49c4e0b80 1 client.36395 shutdown 2024-05-27T07:12:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.868+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 msgr2=0x56404ab45510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.868+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 0x56404ab45510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c8c0 tx=0x7ff4780a92f0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 shutdown_connections 2024-05-27T07:12:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404ab55600 0x56404ab57a20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404a9bce40 0x56404ab45510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3799820591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404ab3bdb0 0x56404ab45a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.869+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 >> 172.21.15.16:0/3799820591 conn(0x56404aa1cfc0 msgr2=0x56404ab36580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.870+0000 7ff470c00700 0 client.36395 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.871+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 shutdown_connections 2024-05-27T07:12:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.871+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3799820591 wait complete. 2024-05-27T07:12:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.872+0000 7ff49c4e0b80 20 client.36395 trim_cache size 0 max 16384 2024-05-27T07:12:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.906+0000 7ff49c4e0b80 1 Processor -- start 2024-05-27T07:12:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.906+0000 7ff49c4e0b80 1 -- start start 2024-05-27T07:12:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.907+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab3b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.907+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.907+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.908+0000 7ff486000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.908+0000 7ff486000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:53244/0 (socket says 172.21.15.16:53244) 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.908+0000 7ff487400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.908+0000 7ff487400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab51b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:34902/0 (socket says 172.21.15.16:34902) 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.908+0000 7ff487400700 1 -- 172.21.15.16:0/3509456511 learned_addr learned my addr 172.21.15.16:0/3509456511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.909+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b6c70 con 0x56404a9bce40 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.909+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b87b0 con 0x56404ab3bdb0 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.909+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b7340 con 0x56404ab41d30 2024-05-27T07:12:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.909+0000 7ff486a00700 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab3b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.910+0000 7ff486000700 1 -- 172.21.15.16:0/3509456511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 msgr2=0x56404ab51b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.910+0000 7ff486000700 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab51b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.910+0000 7ff486000700 1 -- 172.21.15.16:0/3509456511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 msgr2=0x56404ab3b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.910+0000 7ff486000700 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab3b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.910+0000 7ff486000700 1 -- 172.21.15.16:0/3509456511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56404ab38310 con 0x56404ab3bdb0 2024-05-27T07:12:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.911+0000 7ff486000700 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56404ab373e0 tx=0x7ff474009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ac1940e0abf608 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.911+0000 7ff473400700 1 -- 172.21.15.16:0/3509456511 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474036070 con 0x56404ab3bdb0 2024-05-27T07:12:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.911+0000 7ff473400700 1 -- 172.21.15.16:0/3509456511 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474003e70 con 0x56404ab3bdb0 2024-05-27T07:12:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.911+0000 7ff473400700 1 -- 172.21.15.16:0/3509456511 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47400faa0 con 0x56404ab3bdb0 2024-05-27T07:12:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 msgr2=0x56404ab44830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56404ab373e0 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 shutdown_connections 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab41d30 0x56404ab51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab3b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/3509456511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab3bdb0 0x56404ab44830 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.913+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 >> 172.21.15.16:0/3509456511 conn(0x56404aa1cfc0 msgr2=0x56404ab3f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.914+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 shutdown_connections 2024-05-27T07:12:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.914+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/3509456511 wait complete. 2024-05-27T07:12:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.915+0000 7ff49c4e0b80 1 -- start start 2024-05-27T07:12:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.915+0000 7ff49c4e0b80 1 Processor -- start 2024-05-27T07:12:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.916+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab5acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab57610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff486a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab5acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff49c4e0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab57b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b80e0 con 0x56404a9bce40 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b7a10 con 0x56404ab41d30 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.917+0000 7ff49c4e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4881b4bd0 con 0x56404ab3bdb0 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff486000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab57610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff486a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab5acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55882/0 (socket says 172.21.15.16:55882) 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff486a00700 1 -- 172.21.15.16:0/2983997976 learned_addr learned my addr 172.21.15.16:0/2983997976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff487400700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab57b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff49c4e0b80 10 client.0 _ll_register_callbacks cb 0x56404aa03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff487400700 1 -- 172.21.15.16:0/2983997976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 msgr2=0x56404ab57610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff487400700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab57610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff487400700 1 -- 172.21.15.16:0/2983997976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 msgr2=0x56404ab5acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.918+0000 7ff487400700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab5acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.919+0000 7ff487400700 1 -- 172.21.15.16:0/2983997976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56404ab58250 con 0x56404ab41d30 2024-05-27T07:12:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.919+0000 7ff486000700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab57610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.919+0000 7ff470c00700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.919+0000 7ff487400700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab57b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff48000ae80 tx=0x7ff480012510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d5774fb7c63960 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.923+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480012ca0 con 0x56404ab41d30 2024-05-27T07:12:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.923+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474003e70 con 0x56404ab41d30 2024-05-27T07:12:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.923+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff480012aa0 con 0x56404ab41d30 2024-05-27T07:12:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.923+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff480024050 con 0x56404ab41d30 2024-05-27T07:12:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.925+0000 7ff49c4e0b80 10 client.36401 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.925+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56404abbed20 con 0x56404ab41d30 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474003e70 con 0x56404ab41d30 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff4800043d0 con 0x56404ab41d30 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 10 client.36401 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 10 client.36401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 20 client.36401 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404abbf460 0x56404abc1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 4 client.36401 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.926+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56404aa65570 con 0x56404abbf460 2024-05-27T07:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.927+0000 7ff486a00700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404abbf460 0x56404abc1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.930+0000 7ff486a00700 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404abbf460 0x56404abc1880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231f888049f79c53 server_cookie=9a430afd66034f8e in_seq=0 out_seq=0 2024-05-27T07:12:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.930+0000 7ff470c00700 10 client.36401 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.931+0000 7ff470c00700 1 -- 172.21.15.16:0/2983997976 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56404aa65570 con 0x56404abbf460 2024-05-27T07:12:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.931+0000 7ff470c00700 10 client.36401 handle_command_reply: tid=0 2024-05-27T07:12:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.931+0000 7ff49c4e0b80 1 client.36401 shutdown 2024-05-27T07:12:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.931+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 msgr2=0x56404ab57b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.931+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab57b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff48000ae80 tx=0x7ff480012510 comp rx=0 tx=0).stop 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 shutdown_connections 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x56404abbf460 0x56404abc1880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56404ab41d30 0x56404ab57b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56404a9bce40 0x56404ab5acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 --2- 172.21.15.16:0/2983997976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56404ab3bdb0 0x56404ab57610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 >> 172.21.15.16:0/2983997976 conn(0x56404aa1cfc0 msgr2=0x56404ab3f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff470c00700 0 client.36401 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 shutdown_connections 2024-05-27T07:12:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 1 -- 172.21.15.16:0/2983997976 wait complete. 2024-05-27T07:12:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.932+0000 7ff49c4e0b80 20 client.36401 trim_cache size 0 max 16384 2024-05-27T07:12:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.936+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 msgr2=0x7ff46c0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.936+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 0x7ff46c0b5110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4740037c0 tx=0x7ff47400b540 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.936+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 msgr2=0x7ff4881a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.936+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881a2270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c8f0 tx=0x7ff47800cc00 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 shutdown_connections 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff46c0b2c00 0x7ff46c0b5110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488109860 0x7ff4881ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4881045e0 0x7ff4881a2270 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 --2- 172.21.15.16:0/2198208548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff488108f10 0x7ff4881a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.937+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 >> 172.21.15.16:0/2198208548 conn(0x7ff488100030 msgr2=0x7ff48810a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.939+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 shutdown_connections 2024-05-27T07:12:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:40.939+0000 7ff48da00700 1 -- 172.21.15.16:0/2198208548 wait complete. 2024-05-27T07:12:42.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:41 smithi016 ceph-mon[31415]: pgmap v369: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:42.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:41 smithi164 ceph-mon[35327]: pgmap v369: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:42.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:41 smithi069 ceph-mon[35276]: pgmap v369: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 442 op/s 2024-05-27T07:12:43.086 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:12:43.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:12:43.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:43 smithi016 ceph-mon[31415]: pgmap v370: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 443 op/s 2024-05-27T07:12:44.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:43 smithi164 ceph-mon[35327]: pgmap v370: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 443 op/s 2024-05-27T07:12:44.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:43 smithi069 ceph-mon[35276]: pgmap v370: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 443 op/s 2024-05-27T07:12:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:12:46.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:45 smithi164 ceph-mon[35327]: pgmap v371: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 23 MiB/s wr, 447 op/s 2024-05-27T07:12:46.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:45 smithi016 ceph-mon[31415]: pgmap v371: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 23 MiB/s wr, 447 op/s 2024-05-27T07:12:46.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:45 smithi069 ceph-mon[35276]: pgmap v371: 129 pgs: 129 active+clean; 18 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 23 MiB/s wr, 447 op/s 2024-05-27T07:12:48.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:47 smithi164 ceph-mon[35327]: pgmap v372: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 465 op/s 2024-05-27T07:12:48.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:47 smithi016 ceph-mon[31415]: pgmap v372: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 465 op/s 2024-05-27T07:12:48.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:47 smithi069 ceph-mon[35276]: pgmap v372: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 465 op/s 2024-05-27T07:12:50.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:49 smithi016 ceph-mon[31415]: pgmap v373: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 24 MiB/s wr, 469 op/s 2024-05-27T07:12:50.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:49 smithi069 ceph-mon[35276]: pgmap v373: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 24 MiB/s wr, 469 op/s 2024-05-27T07:12:50.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:49 smithi164 ceph-mon[35327]: pgmap v373: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 24 MiB/s wr, 469 op/s 2024-05-27T07:12:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.771+0000 7fa103e00700 1 -- 172.21.15.16:0/1962325576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104109100 msgr2=0x7fa104109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.771+0000 7fa103e00700 1 --2- 172.21.15.16:0/1962325576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104109100 0x7fa104109510 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009990 tx=0x7fa0f0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 -- 172.21.15.16:0/1962325576 shutdown_connections 2024-05-27T07:12:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 --2- 172.21.15.16:0/1962325576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104102aa0 0x7fa104110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 --2- 172.21.15.16:0/1962325576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104109100 0x7fa104109510 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 --2- 172.21.15.16:0/1962325576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102150 0x7fa104102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 -- 172.21.15.16:0/1962325576 >> 172.21.15.16:0/1962325576 conn(0x7fa104077ee0 msgr2=0x7fa104078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 -- 172.21.15.16:0/1962325576 shutdown_connections 2024-05-27T07:12:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.772+0000 7fa103e00700 1 -- 172.21.15.16:0/1962325576 wait complete. 2024-05-27T07:12:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.775+0000 7fa103e00700 1 Processor -- start 2024-05-27T07:12:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.775+0000 7fa103e00700 1 -- start start 2024-05-27T07:12:50.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.777+0000 7fa103e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 0x7fa10406e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.777+0000 7fa103e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.777+0000 7fa103e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 0x7fa10406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa102a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 0x7fa10406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa103e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa104114280 con 0x7fa104102150 2024-05-27T07:12:50.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa103e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa104114100 con 0x7fa104102aa0 2024-05-27T07:12:50.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa103e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa104114400 con 0x7fa104109100 2024-05-27T07:12:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa102a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 0x7fa10406e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44264/0 (socket says 172.21.15.16:44264) 2024-05-27T07:12:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa102000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa102000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43522/0 (socket says 172.21.15.16:43522) 2024-05-27T07:12:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa102000700 1 -- 172.21.15.16:0/2649691122 learned_addr learned my addr 172.21.15.16:0/2649691122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.778+0000 7fa103400700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 0x7fa10406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa102000700 1 -- 172.21.15.16:0/2649691122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 msgr2=0x7fa10406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa102000700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 0x7fa10406f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa102000700 1 -- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 msgr2=0x7fa10406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa102000700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 0x7fa10406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa102000700 1 -- 172.21.15.16:0/2649691122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa104073060 con 0x7fa104102aa0 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.779+0000 7fa103400700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 0x7fa10406f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.780+0000 7fa102000700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b530 tx=0x7fa0f400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.780+0000 7fa108a00700 1 -- 172.21.15.16:0/2649691122 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f401d070 con 0x7fa104102aa0 2024-05-27T07:12:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.780+0000 7fa108a00700 1 -- 172.21.15.16:0/2649691122 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f4010e00 con 0x7fa104102aa0 2024-05-27T07:12:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.780+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104073350 con 0x7fa104102aa0 2024-05-27T07:12:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.781+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1040738e0 con 0x7fa104102aa0 2024-05-27T07:12:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.781+0000 7fa108a00700 1 -- 172.21.15.16:0/2649691122 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4016340 con 0x7fa104102aa0 2024-05-27T07:12:50.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.788+0000 7fa108a00700 1 -- 172.21.15.16:0/2649691122 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa0f4020070 con 0x7fa104102aa0 2024-05-27T07:12:50.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.790+0000 7fa108a00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 0x7fa0e80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.791+0000 7fa102a00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 0x7fa0e80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.792+0000 7fa108a00700 1 -- 172.21.15.16:0/2649691122 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7fa0f40e6880 con 0x7fa104102aa0 2024-05-27T07:12:50.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.853+0000 7fa102a00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 0x7fa0e80b5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa10406feb0 tx=0x7fa0f000b540 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.857+0000 7fa11887eb80 1 Processor -- start 2024-05-27T07:12:50.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.861+0000 7fa11887eb80 1 -- start start 2024-05-27T07:12:50.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.861+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93592a70 0x55ac936267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.861+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.861+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac93632760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa11887eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac93553ec0 con 0x55ac93592a70 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa11887eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ac93543a60 con 0x55ac93630340 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa11887eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac93547aa0 con 0x55ac934a6ec0 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa102000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa102000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60726/0 (socket says 172.21.15.16:60726) 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.862+0000 7fa102000700 1 -- 172.21.15.16:0/2281811330 learned_addr learned my addr 172.21.15.16:0/2281811330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa103400700 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac93632760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102a00700 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93592a70 0x55ac936267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102000700 1 -- 172.21.15.16:0/2281811330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 msgr2=0x55ac93632760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102000700 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac93632760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102000700 1 -- 172.21.15.16:0/2281811330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93592a70 msgr2=0x55ac936267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102000700 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93592a70 0x55ac936267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.863+0000 7fa102000700 1 -- 172.21.15.16:0/2281811330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9350c4a0 con 0x55ac934a6ec0 2024-05-27T07:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.864+0000 7fa102000700 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bc40 tx=0x7fa0f40ad2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8faac1e3e8c2740 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.864+0000 7fa0f8c00700 1 -- 172.21.15.16:0/2281811330 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40adf50 con 0x55ac934a6ec0 2024-05-27T07:12:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.864+0000 7fa0f8c00700 1 -- 172.21.15.16:0/2281811330 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f40ad850 con 0x55ac934a6ec0 2024-05-27T07:12:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.864+0000 7fa0f8c00700 1 -- 172.21.15.16:0/2281811330 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40b6220 con 0x55ac934a6ec0 2024-05-27T07:12:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.867+0000 7fa11887eb80 1 -- 172.21.15.16:0/2281811330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 msgr2=0x55ac93626d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.867+0000 7fa11887eb80 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bc40 tx=0x7fa0f40ad2a0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.868+0000 7fa11887eb80 1 -- 172.21.15.16:0/2281811330 shutdown_connections 2024-05-27T07:12:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.868+0000 7fa11887eb80 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac93632760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.868+0000 7fa11887eb80 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93592a70 0x55ac936267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.868+0000 7fa11887eb80 1 --2- 172.21.15.16:0/2281811330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93626d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.868+0000 7fa11887eb80 1 -- 172.21.15.16:0/2281811330 >> 172.21.15.16:0/2281811330 conn(0x55ac93540410 msgr2=0x55ac93621790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.870+0000 7fa11887eb80 1 -- 172.21.15.16:0/2281811330 shutdown_connections 2024-05-27T07:12:50.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.870+0000 7fa11887eb80 1 -- 172.21.15.16:0/2281811330 wait complete. 2024-05-27T07:12:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.871+0000 7fa11887eb80 1 -- start start 2024-05-27T07:12:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.872+0000 7fa11887eb80 1 Processor -- start 2024-05-27T07:12:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.877+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac934a6ec0 0x55ac9362d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.877+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9362dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa11887eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac9344a020 con 0x55ac934a6ec0 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa11887eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ac934e47e0 con 0x55ac93630340 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa11887eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac93632e80 con 0x55ac93592a70 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa102000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa11887eb80 10 client.0 _ll_register_callbacks cb 0x55ac934edcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa102000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60736/0 (socket says 172.21.15.16:60736) 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa102000700 1 -- 172.21.15.16:0/3991844522 learned_addr learned my addr 172.21.15.16:0/3991844522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.878+0000 7fa103400700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9362dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102a00700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac934a6ec0 0x55ac9362d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102000700 1 -- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 msgr2=0x55ac9362dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102000700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9362dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102000700 1 -- 172.21.15.16:0/3991844522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac934a6ec0 msgr2=0x55ac9362d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102000700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac934a6ec0 0x55ac9362d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.879+0000 7fa102000700 1 -- 172.21.15.16:0/3991844522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac93493950 con 0x55ac93592a70 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.881+0000 7fa102000700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40aaac0 tx=0x7fa0f40e3d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1bb99825fcaeab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.882+0000 7fa0ed600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:12:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40adf50 con 0x55ac93592a70 2024-05-27T07:12:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7fa0f40b6430 con 0x55ac93592a70 2024-05-27T07:12:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f40ad850 con 0x55ac93592a70 2024-05-27T07:12:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0f40b66b0 con 0x55ac93592a70 2024-05-27T07:12:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa11887eb80 10 client.26341 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.883+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ac935140e0 con 0x55ac93592a70 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.884+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f40ad850 con 0x55ac93592a70 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f4010970 con 0x55ac93592a70 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 10 client.26341 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 20 client.26341 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac9363fc30 0x55ac93642050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 4 client.26341 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.887+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac93555e10 con 0x55ac9363fc30 2024-05-27T07:12:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.888+0000 7fa102a00700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac9363fc30 0x55ac93642050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.891+0000 7fa0ed600700 10 client.26341 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.891+0000 7fa102a00700 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac9363fc30 0x55ac93642050 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40ea7196bc1d44a server_cookie=7fa443f5f4482c in_seq=0 out_seq=0 2024-05-27T07:12:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.893+0000 7fa0ed600700 1 -- 172.21.15.16:0/3991844522 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac93555e10 con 0x55ac9363fc30 2024-05-27T07:12:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.893+0000 7fa0ed600700 10 client.26341 handle_command_reply: tid=0 2024-05-27T07:12:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.893+0000 7fa11887eb80 1 client.26341 shutdown 2024-05-27T07:12:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.893+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 msgr2=0x55ac9362d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40aaac0 tx=0x7fa0f40e3d30 comp rx=0 tx=0).stop 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 shutdown_connections 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac9363fc30 0x55ac93642050 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9362dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac934a6ec0 0x55ac9362d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 --2- 172.21.15.16:0/3991844522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93592a70 0x55ac9362d810 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40aaac0 tx=0x7fa0f40e3d30 comp rx=0 tx=0).stop 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa0ed600700 0 client.26341 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.894+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 >> 172.21.15.16:0/3991844522 conn(0x55ac93540410 msgr2=0x55ac93621380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.895+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 shutdown_connections 2024-05-27T07:12:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.895+0000 7fa11887eb80 1 -- 172.21.15.16:0/3991844522 wait complete. 2024-05-27T07:12:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.896+0000 7fa11887eb80 20 client.26341 trim_cache size 0 max 16384 2024-05-27T07:12:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.941+0000 7fa11887eb80 1 Processor -- start 2024-05-27T07:12:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.941+0000 7fa11887eb80 1 -- start start 2024-05-27T07:12:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.943+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.943+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9363c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac9362efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa11887eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e400ed10 con 0x55ac93628930 2024-05-27T07:12:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa11887eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4010850 con 0x55ac93630340 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa11887eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e400f3e0 con 0x55ac934a6ec0 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:60752/0 (socket says 172.21.15.16:60752) 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 -- 172.21.15.16:0/4051506782 learned_addr learned my addr 172.21.15.16:0/4051506782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 -- 172.21.15.16:0/4051506782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 msgr2=0x55ac9363c150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9363c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 -- 172.21.15.16:0/4051506782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 msgr2=0x55ac9362efe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac9362efe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.944+0000 7fa102a00700 1 -- 172.21.15.16:0/4051506782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9362fb70 con 0x55ac934a6ec0 2024-05-27T07:12:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.945+0000 7fa102a00700 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005ed0 tx=0x7fa0f001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62844f0e2eb4b158 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.945+0000 7fa0efe00700 1 -- 172.21.15.16:0/4051506782 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0039070 con 0x55ac934a6ec0 2024-05-27T07:12:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.945+0000 7fa0efe00700 1 -- 172.21.15.16:0/4051506782 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f0003e50 con 0x55ac934a6ec0 2024-05-27T07:12:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.946+0000 7fa0efe00700 1 -- 172.21.15.16:0/4051506782 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000fcf0 con 0x55ac934a6ec0 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.947+0000 7fa11887eb80 1 -- 172.21.15.16:0/4051506782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 msgr2=0x55ac93625bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.951+0000 7fa11887eb80 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005ed0 tx=0x7fa0f001e000 comp rx=0 tx=0).stop 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.951+0000 7fa11887eb80 1 -- 172.21.15.16:0/4051506782 shutdown_connections 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.951+0000 7fa11887eb80 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac9362efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.951+0000 7fa11887eb80 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac934a6ec0 0x55ac93625bd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.951+0000 7fa11887eb80 1 --2- 172.21.15.16:0/4051506782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac93630340 0x55ac9363c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.952+0000 7fa11887eb80 1 -- 172.21.15.16:0/4051506782 >> 172.21.15.16:0/4051506782 conn(0x55ac93540410 msgr2=0x55ac9362e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.952+0000 7fa11887eb80 1 -- 172.21.15.16:0/4051506782 shutdown_connections 2024-05-27T07:12:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.952+0000 7fa11887eb80 1 -- 172.21.15.16:0/4051506782 wait complete. 2024-05-27T07:12:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.953+0000 7fa11887eb80 1 -- start start 2024-05-27T07:12:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.954+0000 7fa11887eb80 1 Processor -- start 2024-05-27T07:12:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.955+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.955+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac93643bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.956+0000 7fa102a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.956+0000 7fa11887eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 0x55ac93644190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.956+0000 7fa102a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:43568/0 (socket says 172.21.15.16:43568) 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.956+0000 7fa102a00700 1 -- 172.21.15.16:0/164956176 learned_addr learned my addr 172.21.15.16:0/164956176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.957+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4010180 con 0x55ac93628930 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.957+0000 7fa103400700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 0x55ac93644190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.957+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e400fab0 con 0x55ac934a6ec0 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.957+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e400ccd0 con 0x55ac93630340 2024-05-27T07:12:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.957+0000 7fa11887eb80 10 client.0 _ll_register_callbacks cb 0x55ac934edcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:12:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa102a00700 1 -- 172.21.15.16:0/164956176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 msgr2=0x55ac93644190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa102a00700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 0x55ac93644190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa102a00700 1 -- 172.21.15.16:0/164956176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 msgr2=0x55ac93643bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa102a00700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac93643bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa102a00700 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac93644930 con 0x55ac934a6ec0 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f0003e50 con 0x55ac934a6ec0 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.958+0000 7fa103400700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 0x55ac93644190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.959+0000 7fa0ed600700 10 client.36410 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:12:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.959+0000 7fa102a00700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a709c3ea498865 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.960+0000 7fa0ed600700 1 -- 172.21.15.16:0/164956176 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0039070 con 0x55ac934a6ec0 2024-05-27T07:12:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.960+0000 7fa0ed600700 1 -- 172.21.15.16:0/164956176 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7fa0f000f730 con 0x55ac934a6ec0 2024-05-27T07:12:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.961+0000 7fa0ed600700 1 -- 172.21.15.16:0/164956176 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0f0039070 con 0x55ac934a6ec0 2024-05-27T07:12:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.961+0000 7fa11887eb80 10 client.36410 fetch_fsmap learned FSMap version 31 2024-05-27T07:12:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.961+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55ac936ab3a0 con 0x55ac934a6ec0 2024-05-27T07:12:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa0ed600700 1 -- 172.21.15.16:0/164956176 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f0022070 con 0x55ac934a6ec0 2024-05-27T07:12:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 10 client.36410 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:12:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 10 client.36410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:12:50.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 20 client.36410 populate_metadata read hostname 'smithi016' 2024-05-27T07:12:50.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac936ab8c0 0x55ac936adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:50.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 4 client.36410 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:12:50.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.962+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac9354f5f0 con 0x55ac936ab8c0 2024-05-27T07:12:50.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.963+0000 7fa102000700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac936ab8c0 0x55ac936adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.963+0000 7fa102000700 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac936ab8c0 0x55ac936adce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89cbe57f2b9ce2d server_cookie=8086075bbc844253 in_seq=0 out_seq=0 2024-05-27T07:12:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.964+0000 7fa0ed600700 10 client.36410 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:50.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.965+0000 7fa0ed600700 1 -- 172.21.15.16:0/164956176 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac9354f5f0 con 0x55ac936ab8c0 2024-05-27T07:12:50.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.965+0000 7fa0ed600700 10 client.36410 handle_command_reply: tid=0 2024-05-27T07:12:50.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.965+0000 7fa11887eb80 1 client.36410 shutdown 2024-05-27T07:12:50.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 msgr2=0x55ac93645840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f0009310 comp rx=0 tx=0).stop 2024-05-27T07:12:50.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 shutdown_connections 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x55ac936ab8c0 0x55ac936adce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac93630340 0x55ac93644190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac934a6ec0 0x55ac93645840 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f0009310 comp rx=0 tx=0).stop 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 --2- 172.21.15.16:0/164956176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac93628930 0x55ac93643bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 >> 172.21.15.16:0/164956176 conn(0x55ac93540410 msgr2=0x55ac93625930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.967+0000 7fa0ed600700 0 client.36410 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.968+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 shutdown_connections 2024-05-27T07:12:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.968+0000 7fa11887eb80 1 -- 172.21.15.16:0/164956176 wait complete. 2024-05-27T07:12:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.970+0000 7fa11887eb80 20 client.36410 trim_cache size 0 max 16384 2024-05-27T07:12:50.961 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:12:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-27T07:12:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "129ba9b2-8164-4fce-9e1c-6634dc07cc7d", 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.971+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 msgr2=0x7fa0e80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.971+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 0x7fa0e80b5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa10406feb0 tx=0x7fa0f000b540 comp rx=0 tx=0).stop 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.971+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 msgr2=0x7fa10406eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.971+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b530 tx=0x7fa0f400b840 comp rx=0 tx=0).stop 2024-05-27T07:12:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 shutdown_connections 2024-05-27T07:12:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fa0e80b2af0 0x7fa0e80b5000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa104109100 0x7fa10406f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa104102150 0x7fa10406e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 --2- 172.21.15.16:0/2649691122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa104102aa0 0x7fa10406eba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.974+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 >> 172.21.15.16:0/2649691122 conn(0x7fa104077ee0 msgr2=0x7fa1040ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.976+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 shutdown_connections 2024-05-27T07:12:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:50.976+0000 7fa103e00700 1 -- 172.21.15.16:0/2649691122 wait complete. 2024-05-27T07:12:52.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:51 smithi016 ceph-mon[31415]: pgmap v374: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 441 op/s 2024-05-27T07:12:52.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:51 smithi069 ceph-mon[35276]: pgmap v374: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 441 op/s 2024-05-27T07:12:52.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:12:52.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:51 smithi164 ceph-mon[35327]: pgmap v374: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 441 op/s 2024-05-27T07:12:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:12:53.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:52 smithi016 ceph-mon[31415]: scrub queued for path: / 2024-05-27T07:12:53.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:52 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:53.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:52 smithi016 ceph-mon[31415]: scrub summary: active paths [/] 2024-05-27T07:12:53.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:52 smithi016 ceph-mon[31415]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:52 smithi164 ceph-mon[35327]: scrub queued for path: / 2024-05-27T07:12:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:52 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:52 smithi164 ceph-mon[35327]: scrub summary: active paths [/] 2024-05-27T07:12:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:52 smithi164 ceph-mon[35327]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:53.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:52 smithi069 ceph-mon[35276]: scrub queued for path: / 2024-05-27T07:12:53.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:52 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:52 smithi069 ceph-mon[35276]: scrub summary: active paths [/] 2024-05-27T07:12:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:52 smithi069 ceph-mon[35276]: scrub summary: idle+waiting paths [/] 2024-05-27T07:12:54.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:53 smithi016 ceph-mon[31415]: pgmap v375: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 27 MiB/s wr, 537 op/s 2024-05-27T07:12:54.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:53 smithi164 ceph-mon[35327]: pgmap v375: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 27 MiB/s wr, 537 op/s 2024-05-27T07:12:54.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:53 smithi069 ceph-mon[35276]: pgmap v375: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 27 MiB/s wr, 537 op/s 2024-05-27T07:12:55.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:54 smithi016 ceph-mon[31415]: scrub summary: idle 2024-05-27T07:12:55.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:54 smithi164 ceph-mon[35327]: scrub summary: idle 2024-05-27T07:12:55.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:54 smithi069 ceph-mon[35276]: scrub summary: idle 2024-05-27T07:12:56.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:55 smithi164 ceph-mon[35327]: pgmap v376: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 21 MiB/s wr, 423 op/s 2024-05-27T07:12:56.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:55 smithi016 ceph-mon[31415]: pgmap v376: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 21 MiB/s wr, 423 op/s 2024-05-27T07:12:56.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:55 smithi069 ceph-mon[35276]: pgmap v376: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 21 MiB/s wr, 423 op/s 2024-05-27T07:12:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:57 smithi016 ceph-mon[31415]: pgmap v377: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 27 MiB/s wr, 521 op/s 2024-05-27T07:12:58.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:57 smithi164 ceph-mon[35327]: pgmap v377: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 27 MiB/s wr, 521 op/s 2024-05-27T07:12:58.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:57 smithi069 ceph-mon[35276]: pgmap v377: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 27 MiB/s wr, 521 op/s 2024-05-27T07:12:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.968+0000 7ff85b400700 1 -- 172.21.15.16:0/3245246717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85411d920 msgr2=0x7ff85411fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.968+0000 7ff85b400700 1 --2- 172.21.15.16:0/3245246717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85411d920 0x7ff85411fd40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b0a0 tx=0x7ff84c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.969+0000 7ff85b400700 1 -- 172.21.15.16:0/3245246717 shutdown_connections 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.969+0000 7ff85b400700 1 --2- 172.21.15.16:0/3245246717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85411d920 0x7ff85411fd40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.969+0000 7ff85b400700 1 --2- 172.21.15.16:0/3245246717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85410a920 0x7ff85410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.969+0000 7ff85b400700 1 --2- 172.21.15.16:0/3245246717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410b270 0x7ff85411d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.969+0000 7ff85b400700 1 -- 172.21.15.16:0/3245246717 >> 172.21.15.16:0/3245246717 conn(0x7ff85406f4c0 msgr2=0x7ff85406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.970+0000 7ff85b400700 1 -- 172.21.15.16:0/3245246717 shutdown_connections 2024-05-27T07:12:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.970+0000 7ff85b400700 1 -- 172.21.15.16:0/3245246717 wait complete. 2024-05-27T07:12:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.972+0000 7ff85b400700 1 Processor -- start 2024-05-27T07:12:59.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.972+0000 7ff85b400700 1 -- start start 2024-05-27T07:12:59.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.973+0000 7ff85b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 0x7ff8541af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:59.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.973+0000 7ff85b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 0x7ff8541afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:59.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.973+0000 7ff85b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:59.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.974+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 0x7ff8541af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:59.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.974+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 0x7ff8541af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42310/0 (socket says 172.21.15.16:42310) 2024-05-27T07:12:59.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.974+0000 7ff853e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 0x7ff8541afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff853e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 0x7ff8541afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41074/0 (socket says 172.21.15.16:41074) 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff859400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff859400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:56406/0 (socket says 172.21.15.16:56406) 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff858a00700 1 -- 172.21.15.16:0/1958336650 learned_addr learned my addr 172.21.15.16:0/1958336650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff854123160 con 0x7ff85410b270 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff854122fe0 con 0x7ff85410a920 2024-05-27T07:12:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.975+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541232e0 con 0x7ff85411d920 2024-05-27T07:12:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.976+0000 7ff859400700 1 -- 172.21.15.16:0/1958336650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 msgr2=0x7ff8541af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.976+0000 7ff859400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 0x7ff8541af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.977+0000 7ff859400700 1 -- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 msgr2=0x7ff8541afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:12:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.977+0000 7ff859400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 0x7ff8541afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:12:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.977+0000 7ff859400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8541a9fd0 con 0x7ff85411d920 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.977+0000 7ff859400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff854112b90 tx=0x7ff84c004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.978+0000 7ff851600700 1 -- 172.21.15.16:0/1958336650 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c021070 con 0x7ff85411d920 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.978+0000 7ff851600700 1 -- 172.21.15.16:0/1958336650 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff84c007b80 con 0x7ff85411d920 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.978+0000 7ff851600700 1 -- 172.21.15.16:0/1958336650 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c005040 con 0x7ff85411d920 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.978+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8541aa260 con 0x7ff85411d920 2024-05-27T07:12:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.979+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8541aa630 con 0x7ff85411d920 2024-05-27T07:12:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.988+0000 7ff851600700 1 -- 172.21.15.16:0/1958336650 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff84c024070 con 0x7ff85411d920 2024-05-27T07:12:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.991+0000 7ff851600700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 0x7ff8380b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:12:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.991+0000 7ff851600700 1 -- 172.21.15.16:0/1958336650 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff84c00f020 con 0x7ff85411d920 2024-05-27T07:12:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.994+0000 7ff858a00700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 0x7ff8380b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:12:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:12:59.995+0000 7ff858a00700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 0x7ff8380b5030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff844005950 tx=0x7ff84400a400 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.020+0000 7ff869f08b80 1 Processor -- start 2024-05-27T07:13:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.021+0000 7ff869f08b80 1 -- start start 2024-05-27T07:13:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.022+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.022+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.023+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.023+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.023+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42314/0 (socket says 172.21.15.16:42314) 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff869f08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55771101ae40 con 0x5577110ece70 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff869f08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55771100a9e0 con 0x5577110599f0 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff869f08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55771100ea20 con 0x557710f6de40 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff858a00700 1 -- 172.21.15.16:0/1683558072 learned_addr learned my addr 172.21.15.16:0/1683558072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff853e00700 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.024+0000 7ff859400700 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 -- 172.21.15.16:0/1683558072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 msgr2=0x5577110ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 -- 172.21.15.16:0/1683558072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 msgr2=0x5577110f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 -- 172.21.15.16:0/1683558072 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557710fd3420 con 0x5577110599f0 2024-05-27T07:13:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.025+0000 7ff858a00700 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ff60 tx=0x7ff844012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b6392553eb6a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.026+0000 7ff84a000700 1 -- 172.21.15.16:0/1683558072 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844016070 con 0x5577110599f0 2024-05-27T07:13:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.026+0000 7ff84a000700 1 -- 172.21.15.16:0/1683558072 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff844025470 con 0x5577110599f0 2024-05-27T07:13:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.026+0000 7ff84a000700 1 -- 172.21.15.16:0/1683558072 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84401b620 con 0x5577110599f0 2024-05-27T07:13:00.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.029+0000 7ff869f08b80 1 -- 172.21.15.16:0/1683558072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 msgr2=0x5577110ec330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.029+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ff60 tx=0x7ff844012000 comp rx=0 tx=0).stop 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 -- 172.21.15.16:0/1683558072 shutdown_connections 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110599f0 0x5577110ec330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1683558072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 -- 172.21.15.16:0/1683558072 >> 172.21.15.16:0/1683558072 conn(0x557711007390 msgr2=0x5577110e84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 -- 172.21.15.16:0/1683558072 shutdown_connections 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.030+0000 7ff869f08b80 1 -- 172.21.15.16:0/1683558072 wait complete. 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.031+0000 7ff869f08b80 1 -- start start 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.032+0000 7ff869f08b80 1 Processor -- start 2024-05-27T07:13:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.032+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.033+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x5577110f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff869f08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557710f10fa0 con 0x5577110599f0 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff869f08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557710fab760 con 0x5577110ece70 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff869f08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5577110f9b80 con 0x557710f6de40 2024-05-27T07:13:00.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff869f08b80 10 client.0 _ll_register_callbacks cb 0x557710fb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.034+0000 7ff858a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.035+0000 7ff859400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x5577110f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.035+0000 7ff859400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x5577110f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42324/0 (socket says 172.21.15.16:42324) 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.035+0000 7ff859400700 1 -- 172.21.15.16:0/1823502328 learned_addr learned my addr 172.21.15.16:0/1823502328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.035+0000 7ff853e00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 -- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 msgr2=0x5577110f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x5577110f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 -- 172.21.15.16:0/1823502328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 msgr2=0x5577110f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 -- 172.21.15.16:0/1823502328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557710f5a8d0 con 0x557710f6de40 2024-05-27T07:13:00.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff858a00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f6660 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ff30 tx=0x7ff844010b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1298e7611e99d9c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:00.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.036+0000 7ff83ea00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:13:00.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.037+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84401c620 con 0x557710f6de40 2024-05-27T07:13:00.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.037+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff84401c7c0 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.038+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff844025470 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.038+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff84401ca00 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.038+0000 7ff869f08b80 10 client.26356 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.038+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557710fdb060 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff844025470 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff844005020 con 0x557710f6de40 2024-05-27T07:13:00.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff869f08b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff869f08b80 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff869f08b80 20 client.26356 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.039+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111179e0 0x557711119e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.040+0000 7ff869f08b80 4 client.26356 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.040+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771101cd90 con 0x5577111179e0 2024-05-27T07:13:00.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.040+0000 7ff853e00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111179e0 0x557711119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.042+0000 7ff853e00700 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111179e0 0x557711119e00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1968e5ab05d94025 server_cookie=496063b188e4469 in_seq=0 out_seq=0 2024-05-27T07:13:00.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.042+0000 7ff83ea00700 10 client.26356 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.046+0000 7ff83ea00700 1 -- 172.21.15.16:0/1823502328 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55771101cd90 con 0x5577111179e0 2024-05-27T07:13:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.046+0000 7ff83ea00700 10 client.26356 handle_command_reply: tid=0 2024-05-27T07:13:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.047+0000 7ff869f08b80 1 client.26356 shutdown 2024-05-27T07:13:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 msgr2=0x5577110f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f6660 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ff30 tx=0x7ff844010b70 comp rx=0 tx=0).stop 2024-05-27T07:13:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 shutdown_connections 2024-05-27T07:13:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111179e0 0x557711119e00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x5577110f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f6660 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 --2- 172.21.15.16:0/1823502328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.048+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 >> 172.21.15.16:0/1823502328 conn(0x557711007390 msgr2=0x5577110ec310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.049+0000 7ff83ea00700 0 client.26356 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.049+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 shutdown_connections 2024-05-27T07:13:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.050+0000 7ff869f08b80 1 -- 172.21.15.16:0/1823502328 wait complete. 2024-05-27T07:13:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.050+0000 7ff869f08b80 20 client.26356 trim_cache size 0 max 16384 2024-05-27T07:13:00.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 Processor -- start 2024-05-27T07:13:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 -- start start 2024-05-27T07:13:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x557711113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541b6af0 con 0x5577110599f0 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541b8630 con 0x5577110ece70 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.096+0000 7ff869f08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541b71c0 con 0x557710f6de40 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.097+0000 7ff858a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.097+0000 7ff858a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:56426/0 (socket says 172.21.15.16:56426) 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.097+0000 7ff858a00700 1 -- 172.21.15.16:0/3255112645 learned_addr learned my addr 172.21.15.16:0/3255112645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.097+0000 7ff859400700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x557711113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff853e00700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff858a00700 1 -- 172.21.15.16:0/3255112645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 msgr2=0x557711113cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff858a00700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x557711113cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff858a00700 1 -- 172.21.15.16:0/3255112645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 msgr2=0x5577110f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff858a00700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff858a00700 1 -- 172.21.15.16:0/3255112645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771111a2a0 con 0x557710f6de40 2024-05-27T07:13:00.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.098+0000 7ff853e00700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:13:00.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.100+0000 7ff858a00700 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff844010710 tx=0x7ff844005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5baf82b1509e0074 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:00.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.100+0000 7ff849600700 1 -- 172.21.15.16:0/3255112645 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84403e020 con 0x557710f6de40 2024-05-27T07:13:00.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.100+0000 7ff849600700 1 -- 172.21.15.16:0/3255112645 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8440056f0 con 0x557710f6de40 2024-05-27T07:13:00.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.100+0000 7ff849600700 1 -- 172.21.15.16:0/3255112645 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844016040 con 0x557710f6de40 2024-05-27T07:13:00.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 -- 172.21.15.16:0/3255112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 msgr2=0x5577110f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff844010710 tx=0x7ff844005ed0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 -- 172.21.15.16:0/3255112645 shutdown_connections 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577110ece70 0x557711113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557710f6de40 0x5577110f1e30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3255112645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110599f0 0x5577110f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.103+0000 7ff869f08b80 1 -- 172.21.15.16:0/3255112645 >> 172.21.15.16:0/3255112645 conn(0x557711007390 msgr2=0x5577110eded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.104+0000 7ff869f08b80 1 -- 172.21.15.16:0/3255112645 shutdown_connections 2024-05-27T07:13:00.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.104+0000 7ff869f08b80 1 -- 172.21.15.16:0/3255112645 wait complete. 2024-05-27T07:13:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.105+0000 7ff869f08b80 1 -- start start 2024-05-27T07:13:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.105+0000 7ff869f08b80 1 Processor -- start 2024-05-27T07:13:00.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.106+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577110599f0 0x5577110f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541b7f60 con 0x5577110ece70 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541b7890 con 0x557710f6de40 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff854119890 con 0x5577110599f0 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff858a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:42336/0 (socket says 172.21.15.16:42336) 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff858a00700 1 -- 172.21.15.16:0/3901261953 learned_addr learned my addr 172.21.15.16:0/3901261953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff859400700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.107+0000 7ff869f08b80 10 client.0 _ll_register_callbacks cb 0x557710fb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:00.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff853e00700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577110599f0 0x5577110f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff859400700 1 -- 172.21.15.16:0/3901261953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577110599f0 msgr2=0x5577110f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff859400700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577110599f0 0x5577110f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff859400700 1 -- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 msgr2=0x557711110380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff859400700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff859400700 1 -- 172.21.15.16:0/3901261953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577110f7450 con 0x5577110ece70 2024-05-27T07:13:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.108+0000 7ff858a00700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:13:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.110+0000 7ff859400700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f1120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006090 tx=0x7ff84c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa153d9008d145a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.110+0000 7ff83ea00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.112+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0aaab0 con 0x5577110ece70 2024-05-27T07:13:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.112+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8440056f0 con 0x5577110ece70 2024-05-27T07:13:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.112+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7ff84c007d30 con 0x5577110ece70 2024-05-27T07:13:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.113+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff84c0b33f0 con 0x5577110ece70 2024-05-27T07:13:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.113+0000 7ff869f08b80 10 client.16488 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.113+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55771110ea00 con 0x5577110ece70 2024-05-27T07:13:00.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.115+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8440056f0 con 0x5577110ece70 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff84c011070 con 0x5577110ece70 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 20 client.16488 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111825e0 0x557711184a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 4 client.16488 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:13:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.117+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557711016570 con 0x5577111825e0 2024-05-27T07:13:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.118+0000 7ff858a00700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111825e0 0x557711184a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.119+0000 7ff858a00700 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111825e0 0x557711184a00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb344bd08020881 server_cookie=d478ae01abf07b5d in_seq=0 out_seq=0 2024-05-27T07:13:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.119+0000 7ff83ea00700 10 client.16488 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.119+0000 7ff83ea00700 1 -- 172.21.15.16:0/3901261953 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557711016570 con 0x5577111825e0 2024-05-27T07:13:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff83ea00700 10 client.16488 handle_command_reply: tid=0 2024-05-27T07:13:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff869f08b80 1 client.16488 shutdown 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 msgr2=0x5577110f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f1120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006090 tx=0x7ff84c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 shutdown_connections 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.120+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5577111825e0 0x557711184a00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577110ece70 0x5577110f1120 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557710f6de40 0x557711110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 --2- 172.21.15.16:0/3901261953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577110599f0 0x5577110f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 >> 172.21.15.16:0/3901261953 conn(0x557711007390 msgr2=0x5577110f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff83ea00700 0 client.16488 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 shutdown_connections 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.121+0000 7ff869f08b80 1 -- 172.21.15.16:0/3901261953 wait complete. 2024-05-27T07:13:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.122+0000 7ff869f08b80 20 client.16488 trim_cache size 0 max 16384 2024-05-27T07:13:00.114 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:13:00.114 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-27T07:13:00.114 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-27T07:13:00.114 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:13:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.123+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 msgr2=0x7ff8380b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.123+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 0x7ff8380b5030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff844005950 tx=0x7ff84400a400 comp rx=0 tx=0).stop 2024-05-27T07:13:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.123+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 msgr2=0x7ff8541a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.123+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff854112b90 tx=0x7ff84c004d50 comp rx=0 tx=0).stop 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 shutdown_connections 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7ff8380b2b20 0x7ff8380b5030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85411d920 0x7ff8541a9920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff85410a920 0x7ff8541af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 --2- 172.21.15.16:0/1958336650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85410b270 0x7ff8541afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 >> 172.21.15.16:0/1958336650 conn(0x7ff85406f4c0 msgr2=0x7ff85411b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 shutdown_connections 2024-05-27T07:13:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:00.124+0000 7ff85b400700 1 -- 172.21.15.16:0/1958336650 wait complete. 2024-05-27T07:13:00.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:12:59 smithi016 ceph-mon[31415]: pgmap v378: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 427 op/s 2024-05-27T07:13:00.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:12:59 smithi164 ceph-mon[35327]: pgmap v378: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 427 op/s 2024-05-27T07:13:00.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:12:59 smithi069 ceph-mon[35276]: pgmap v378: 129 pgs: 129 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 427 op/s 2024-05-27T07:13:02.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:01 smithi164 ceph-mon[35327]: pgmap v379: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 409 op/s 2024-05-27T07:13:02.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:01 smithi016 ceph-mon[31415]: pgmap v379: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 409 op/s 2024-05-27T07:13:02.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:01 smithi069 ceph-mon[35276]: pgmap v379: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 409 op/s 2024-05-27T07:13:02.751 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:13:02.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:13:03.315 INFO:tasks.workunit.client.0.smithi016.stdout:stopping iogen 2024-05-27T07:13:03.316 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'stopping iogen' 2024-05-27T07:13:03.316 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -k 2024-05-27T07:13:03.320 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-05-27T07:13:03.320 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-05-27T07:13:03.597 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:04.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:03 smithi016 ceph-mon[31415]: pgmap v380: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 27 MiB/s wr, 535 op/s 2024-05-27T07:13:04.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:03 smithi164 ceph-mon[35327]: pgmap v380: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 27 MiB/s wr, 535 op/s 2024-05-27T07:13:04.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:03 smithi069 ceph-mon[35276]: pgmap v380: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 27 MiB/s wr, 535 op/s 2024-05-27T07:13:05.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:04 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:13:05.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:04 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:13:05.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:04 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:13:06.181 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T07:13:06.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T07:13:06.231 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:05 smithi016 ceph-mon[31415]: pgmap v381: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 21 MiB/s wr, 432 op/s 2024-05-27T07:13:06.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:05 smithi164 ceph-mon[35327]: pgmap v381: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 21 MiB/s wr, 432 op/s 2024-05-27T07:13:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:05 smithi069 ceph-mon[35276]: pgmap v381: 129 pgs: 129 active+clean; 19 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 21 MiB/s wr, 432 op/s 2024-05-27T07:13:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 -- 172.21.15.16:0/1548762444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b0107de0 msgr2=0x7fd3b01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 --2- 172.21.15.16:0/1548762444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b0107de0 0x7fd3b01081f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0099c0 tx=0x7fd39c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 -- 172.21.15.16:0/1548762444 shutdown_connections 2024-05-27T07:13:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 --2- 172.21.15.16:0/1548762444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0108730 0x7fd3b0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 --2- 172.21.15.16:0/1548762444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b01034b0 0x7fd3b01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 --2- 172.21.15.16:0/1548762444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b0107de0 0x7fd3b01081f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 -- 172.21.15.16:0/1548762444 >> 172.21.15.16:0/1548762444 conn(0x7fd3b00fedb0 msgr2=0x7fd3b01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 -- 172.21.15.16:0/1548762444 shutdown_connections 2024-05-27T07:13:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.283+0000 7fd3b8600700 1 -- 172.21.15.16:0/1548762444 wait complete. 2024-05-27T07:13:07.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:06 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:13:07.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:06 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:13:07.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:06 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.285+0000 7fd3b8600700 1 Processor -- start 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.285+0000 7fd3b8600700 1 -- start start 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0107de0 0x7fd3b019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b0108730 0x7fd3b01a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01143b0 con 0x7fd3b01034b0 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0114230 con 0x7fd3b0108730 2024-05-27T07:13:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b8600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0114530 con 0x7fd3b0107de0 2024-05-27T07:13:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b5c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48616/0 (socket says 172.21.15.16:48616) 2024-05-27T07:13:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.286+0000 7fd3b5c00700 1 -- 172.21.15.16:0/2031807050 learned_addr learned my addr 172.21.15.16:0/2031807050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5200700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0107de0 0x7fd3b019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b6600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b0108730 0x7fd3b01a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 -- 172.21.15.16:0/2031807050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0107de0 msgr2=0x7fd3b019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0107de0 0x7fd3b019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 -- 172.21.15.16:0/2031807050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b0108730 msgr2=0x7fd3b01a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b0108730 0x7fd3b01a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 -- 172.21.15.16:0/2031807050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b01a1e90 con 0x7fd3b01034b0 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.287+0000 7fd3b5c00700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400d950 tx=0x7fd3a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.288+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4016070 con 0x7fd3b01034b0 2024-05-27T07:13:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.288+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b01a2180 con 0x7fd3b01034b0 2024-05-27T07:13:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.288+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01a9aa0 con 0x7fd3b01034b0 2024-05-27T07:13:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.288+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3a400fd20 con 0x7fd3b01034b0 2024-05-27T07:13:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.288+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400e3b0 con 0x7fd3b01034b0 2024-05-27T07:13:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.290+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd390005330 con 0x7fd3b01034b0 2024-05-27T07:13:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.293+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd3a4019070 con 0x7fd3b01034b0 2024-05-27T07:13:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.295+0000 7fd3ae000700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 0x7fd3980b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.295+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9481+0+0 (secure 0 0 0) 0x7fd3a40e2730 con 0x7fd3b01034b0 2024-05-27T07:13:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.296+0000 7fd3b5200700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 0x7fd3980b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.296+0000 7fd3b5200700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 0x7fd3980b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009990 tx=0x7fd39c005fb0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:07.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.314+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3a40a9cf0 con 0x7fd3b01034b0 2024-05-27T07:13:07.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:06 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:13:07.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:06 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:13:07.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:06 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fdfd2400700 time 2024-05-27T07:13:07.065946+0000 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdfe7eeaba6] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: clone() 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: *** Caught signal (Aborted) ** 2024-05-27T07:13:07.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: in thread 7fdfd2400700 thread_name:md_log_replay 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.069+0000 7fdfd2400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fdfd2400700 time 2024-05-27T07:13:07.065946+0000 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdfe7eeaba6] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: clone() 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdfe6a67d20] 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: gsignal() 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: abort() 2024-05-27T07:13:07.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdfe7eeaced] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 19: clone() 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.073+0000 7fdfd2400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: in thread 7fdfd2400700 thread_name:md_log_replay 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdfe6a67d20] 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: gsignal() 2024-05-27T07:13:07.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: abort() 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdfe7eeaced] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 19: clone() 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: -1> 2024-05-27T07:13:07.069+0000 7fdfd2400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fdfd2400700 time 2024-05-27T07:13:07.065946+0000 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:07.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdfe7eeaba6] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: clone() 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 0> 2024-05-27T07:13:07.073+0000 7fdfd2400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: in thread 7fdfd2400700 thread_name:md_log_replay 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdfe6a67d20] 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: gsignal() 2024-05-27T07:13:07.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: abort() 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdfe7eeaced] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 19: clone() 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: -1> 2024-05-27T07:13:07.069+0000 7fdfd2400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fdfd2400700 time 2024-05-27T07:13:07.065946+0000 2024-05-27T07:13:07.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdfe7eeaba6] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: clone() 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 0> 2024-05-27T07:13:07.073+0000 7fdfd2400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: in thread 7fdfd2400700 thread_name:md_log_replay 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdfe6a67d20] 2024-05-27T07:13:07.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2: gsignal() 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 3: abort() 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdfe7eeaced] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdfe7eeadd1] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5584fad9623d] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5584fad81971] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 8: (SnapRealm::check_cache() const+0x13e) [0x5584fad7ffa0] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5584fad80651] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5584fad15347] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5584faa1187e] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5584faa16e94] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5584faa21969] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 14: (MDLog::_replay_thread()+0x1f38) [0x5584fadba48c] 2024-05-27T07:13:07.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5584fadc39cb] 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 16: (Thread::entry_wrapper()+0x43) [0x7fdfe7eb972d] 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdfe7eb9749] 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdfe6a5d1ca] 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 19: clone() 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[65029]: 2024-05-27T07:13:07.348 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:07 smithi164 podman[71352]: 2024-05-27 07:13:07.226986983 +0000 UTC m=+0.036267434 container died 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-05-27T07:13:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:06 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:13:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:06 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:13:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:06 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:13:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.610+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd390005620 con 0x7fd3b01034b0 2024-05-27T07:13:07.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.611+0000 7fd3ae000700 1 -- 172.21.15.16:0/2031807050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7fd3a40a95f0 con 0x7fd3b01034b0 2024-05-27T07:13:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:13:07.603 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/553407450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":553407450},{"type":"v1","addr":"172.21.15.16:6835","nonce":553407450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3811821671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3811821671},{"type":"v1","addr":"172.21.15.16:6837","nonce":3811821671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T06:55:13.518131+0000","modified":"2024-05-27T07:00:52.678585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34478,"mds_2":34451,"mds_3":34421,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/1261534836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1261534836},{"type":"v1","addr":"172.21.15.16:6839","nonce":1261534836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1178863608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1178863608},{"type":"v1","addr":"172.21.15.16:6841","nonce":1178863608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/1294360695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1294360695},{"type":"v1","addr":"172.21.15.164:6833","nonce":1294360695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2731773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2731773245},{"type":"v1","addr":"172.21.15.164:6835","nonce":2731773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/162118671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":162118671},{"type":"v1","addr":"172.21.15.164:6837","nonce":162118671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.164:6839/3910557712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":3910557712},{"type":"v1","addr":"172.21.15.164:6839","nonce":3910557712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":41,"addr":"172.21.15.69:6833/123964468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":123964468},{"type":"v1","addr":"172.21.15.69:6833","nonce":123964468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":37,"addr":"172.21.15.69:6835/4223174404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":4223174404},{"type":"v1","addr":"172.21.15.69:6835","nonce":4223174404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":34,"addr":"172.21.15.69:6837/2624187710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2624187710},{"type":"v1","addr":"172.21.15.69:6837","nonce":2624187710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.69:6839/962503578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":962503578},{"type":"v1","addr":"172.21.15.69:6839","nonce":962503578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34421,24481,34451,34517,34478]},"id":1}]} 2024-05-27T07:13:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 msgr2=0x7fd3980b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 0x7fd3980b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009990 tx=0x7fd39c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 msgr2=0x7fd3b019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400d950 tx=0x7fd3a400dc60 comp rx=0 tx=0).stop 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 shutdown_connections 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd3980b2a70 0x7fd3980b4f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3b0108730 0x7fd3b01a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3b01034b0 0x7fd3b019cfc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 --2- 172.21.15.16:0/2031807050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0107de0 0x7fd3b019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.615+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 >> 172.21.15.16:0/2031807050 conn(0x7fd3b00fedb0 msgr2=0x7fd3b0109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.616+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 shutdown_connections 2024-05-27T07:13:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:07.616+0000 7fd3b8600700 1 -- 172.21.15.16:0/2031807050 wait complete. 2024-05-27T07:13:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-05-27T07:13:08.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-mon[35327]: pgmap v382: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 26 MiB/s wr, 516 op/s 2024-05-27T07:13:08.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:07 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2031807050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:13:08.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:07 smithi016 ceph-mon[31415]: pgmap v382: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 26 MiB/s wr, 516 op/s 2024-05-27T07:13:08.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:07 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2031807050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:13:08.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:07 smithi069 ceph-mon[35276]: pgmap v382: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 26 MiB/s wr, 516 op/s 2024-05-27T07:13:08.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:07 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2031807050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:13:08.422 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-05-27T07:13:08.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T07:13:08.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:13:08.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:09 smithi016 ceph-mon[31415]: pgmap v383: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-05-27T07:13:09.005 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:08 smithi164 podman[71352]: 2024-05-27 07:13:08.628379339 +0000 UTC m=+1.437659838 container remove 4385b4e5d82d5c00edc23dc1fdaced8745764f98dfc05920cd54e1d68f0cd77b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:13:09.005 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:08 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:13:09.006 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:09 smithi164 ceph-mon[35327]: pgmap v383: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-05-27T07:13:09.211 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:09.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:09 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:13:09.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:09 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 5.153s CPU time. 2024-05-27T07:13:09.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:09 smithi069 ceph-mon[35276]: pgmap v383: 129 pgs: 129 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-05-27T07:13:10.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:10 smithi164 ceph-mon[35327]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:13:10.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:10 smithi016 ceph-mon[31415]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:13:10.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:10 smithi069 ceph-mon[35276]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:13:11.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:11 smithi164 ceph-mon[35327]: pgmap v385: 129 pgs: 4 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 109 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 424 op/s 2024-05-27T07:13:11.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:11 smithi016 ceph-mon[31415]: pgmap v385: 129 pgs: 4 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 109 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 424 op/s 2024-05-27T07:13:11.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:11 smithi069 ceph-mon[35276]: pgmap v385: 129 pgs: 4 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 109 active+clean; 19 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 424 op/s 2024-05-27T07:13:13.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:13 smithi164 ceph-mon[35327]: pgmap v386: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 271 op/s 2024-05-27T07:13:13.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:13 smithi016 ceph-mon[31415]: pgmap v386: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 271 op/s 2024-05-27T07:13:13.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:13 smithi069 ceph-mon[35276]: pgmap v386: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 271 op/s 2024-05-27T07:13:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:15 smithi016 ceph-mon[31415]: pgmap v387: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 230 op/s 2024-05-27T07:13:15.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:15 smithi164 ceph-mon[35327]: pgmap v387: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 230 op/s 2024-05-27T07:13:15.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:15 smithi069 ceph-mon[35276]: pgmap v387: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 18 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 230 op/s 2024-05-27T07:13:16.118 DEBUG:teuthology.parallel:result is None 2024-05-27T07:13:16.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T07:13:16.263 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T07:13:16.263 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-27T07:13:16.277 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-27T07:13:16.277 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:13:16.277 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:13:16.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 -- 172.21.15.16:0/2720568155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 msgr2=0x7f2e08109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 --2- 172.21.15.16:0/2720568155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e08109320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009990 tx=0x7f2df0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 -- 172.21.15.16:0/2720568155 shutdown_connections 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 --2- 172.21.15.16:0/2720568155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08109860 0x7f2e081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 --2- 172.21.15.16:0/2720568155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e081049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 --2- 172.21.15.16:0/2720568155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e08109320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 -- 172.21.15.16:0/2720568155 >> 172.21.15.16:0/2720568155 conn(0x7f2e08100030 msgr2=0x7f2e08102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.362+0000 7f2e0e200700 1 -- 172.21.15.16:0/2720568155 shutdown_connections 2024-05-27T07:13:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.363+0000 7f2e0e200700 1 -- 172.21.15.16:0/2720568155 wait complete. 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.364+0000 7f2e0e200700 1 Processor -- start 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.364+0000 7f2e0e200700 1 -- start start 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.365+0000 7f2e0e200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e0819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.365+0000 7f2e0e200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e0819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.365+0000 7f2e0e200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08109860 0x7f2e081a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.366+0000 7f2dfea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e0819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.366+0000 7f2dfea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e0819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38366/0 (socket says 172.21.15.16:38366) 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.366+0000 7f2dfea00700 1 -- 172.21.15.16:0/3867857399 learned_addr learned my addr 172.21.15.16:0/3867857399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.366+0000 7f2e07400700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e0819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08114400 con 0x7f2e08109860 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08114280 con 0x7f2e08108f10 2024-05-27T07:13:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08114580 con 0x7f2e081045e0 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e07400700 1 -- 172.21.15.16:0/3867857399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 msgr2=0x7f2e0819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e07400700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e0819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e07400700 1 -- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08109860 msgr2=0x7f2e081a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e07400700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08109860 0x7f2e081a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.367+0000 7f2e07400700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e081a1d00 con 0x7f2e081045e0 2024-05-27T07:13:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.368+0000 7f2e07400700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e0819ce30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007ec0 tx=0x7f2df800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.369+0000 7f2e04c00700 1 -- 172.21.15.16:0/3867857399 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004db0 con 0x7f2e081045e0 2024-05-27T07:13:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.369+0000 7f2e04c00700 1 -- 172.21.15.16:0/3867857399 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2df8005020 con 0x7f2e081045e0 2024-05-27T07:13:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.369+0000 7f2e04c00700 1 -- 172.21.15.16:0/3867857399 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8015660 con 0x7f2e081045e0 2024-05-27T07:13:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.369+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081a1ff0 con 0x7f2e081045e0 2024-05-27T07:13:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.370+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a9910 con 0x7f2e081045e0 2024-05-27T07:13:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 Processor -- start 2024-05-27T07:13:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 -- start start 2024-05-27T07:13:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67cc5c0 0x563bb67cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb66f1e40 con 0x563bb6644e40 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bb66e19e0 con 0x563bb67d24e0 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.378+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bb66e5a20 con 0x563bb67cc5c0 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.379+0000 7f2e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.379+0000 7f2e07e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50582/0 (socket says 172.21.15.16:50582) 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.379+0000 7f2e07e00700 1 -- 172.21.15.16:0/1866582936 learned_addr learned my addr 172.21.15.16:0/1866582936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.380+0000 7f2e07e00700 1 -- 172.21.15.16:0/1866582936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67cc5c0 msgr2=0x563bb67cc9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:13:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.380+0000 7f2e07e00700 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67cc5c0 0x563bb67cc9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.380+0000 7f2e07e00700 1 -- 172.21.15.16:0/1866582936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 msgr2=0x563bb67cc080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:13:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.380+0000 7f2e07e00700 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67cc080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.380+0000 7f2e07e00700 1 -- 172.21.15.16:0/1866582936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb66aa420 con 0x563bb6644e40 2024-05-27T07:13:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.381+0000 7f2e07e00700 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009820 tx=0x7f2e00009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9f9d51e7a5c6f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.381+0000 7f2dfcc00700 1 -- 172.21.15.16:0/1866582936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00015020 con 0x563bb6644e40 2024-05-27T07:13:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.382+0000 7f2dfcc00700 1 -- 172.21.15.16:0/1866582936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e00011cc0 con 0x563bb6644e40 2024-05-27T07:13:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.382+0000 7f2dfcc00700 1 -- 172.21.15.16:0/1866582936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000bd90 con 0x563bb6644e40 2024-05-27T07:13:16.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.383+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/1866582936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 msgr2=0x563bb67d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.383+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009820 tx=0x7f2e00009b30 comp rx=0 tx=0).stop 2024-05-27T07:13:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.388+0000 7f2e04c00700 1 -- 172.21.15.16:0/3867857399 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2df8015800 con 0x7f2e081045e0 2024-05-27T07:13:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.393+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/1866582936 shutdown_connections 2024-05-27T07:13:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.393+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67cc5c0 0x563bb67cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.393+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb6644e40 0x563bb67d1fa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.393+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/1866582936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.393+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/1866582936 >> 172.21.15.16:0/1866582936 conn(0x563bb66a4fc0 msgr2=0x563bb67be8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.394+0000 7f2e04c00700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 0x7f2dec0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.395+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/1866582936 shutdown_connections 2024-05-27T07:13:16.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.395+0000 7f2dfea00700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 0x7f2dec0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.395+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/1866582936 wait complete. 2024-05-27T07:13:16.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.397+0000 7f2e1cd6db80 1 -- start start 2024-05-27T07:13:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.398+0000 7f2e1cd6db80 1 Processor -- start 2024-05-27T07:13:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.398+0000 7f2e04c00700 1 -- 172.21.15.16:0/3867857399 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f2df80e2430 con 0x7f2e081045e0 2024-05-27T07:13:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.398+0000 7f2dfea00700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 0x7f2dec0b5000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009350 tx=0x7f2df000b540 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.399+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e07400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e07400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36814/0 (socket says 172.21.15.16:36814) 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e07400700 1 -- 172.21.15.16:0/962872703 learned_addr learned my addr 172.21.15.16:0/962872703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb65e7fa0 con 0x563bb67cc5c0 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bb6682760 con 0x563bb67d24e0 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.401+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bb67d9010 con 0x563bb6644e40 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e1cd6db80 10 client.0 _ll_register_callbacks cb 0x563bb668bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2dfea00700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e07400700 1 -- 172.21.15.16:0/962872703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 msgr2=0x563bb67d6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e07400700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67d6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e07400700 1 -- 172.21.15.16:0/962872703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 msgr2=0x563bb67d61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e07400700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67d61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.402+0000 7f2e07400700 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb66318d0 con 0x563bb6644e40 2024-05-27T07:13:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.404+0000 7f2e07400700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2df80a8f70 tx=0x7f2df80a9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1391b49b37be6d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.404+0000 7f2df5600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:13:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.405+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a8d80 con 0x563bb6644e40 2024-05-27T07:13:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.405+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f2df80b1e50 con 0x563bb6644e40 2024-05-27T07:13:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.405+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e00011cc0 con 0x563bb6644e40 2024-05-27T07:13:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.408+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2df80c8050 con 0x563bb6644e40 2024-05-27T07:13:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.408+0000 7f2e1cd6db80 10 client.26377 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.408+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bb66b2060 con 0x563bb6644e40 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e00011cc0 con 0x563bb6644e40 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2df80ad170 con 0x563bb6644e40 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 10 client.26377 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 10 client.26377 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 20 client.26377 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb67e5da0 0x563bb67e8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 4 client.26377 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.409+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb66f3d90 con 0x563bb67e5da0 2024-05-27T07:13:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.412+0000 7f2dfea00700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb67e5da0 0x563bb67e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.413+0000 7f2dfea00700 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb67e5da0 0x563bb67e8240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fe1cb328241a12a server_cookie=991fda638ff873df in_seq=0 out_seq=0 2024-05-27T07:13:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.413+0000 7f2df5600700 10 client.26377 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:13:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.417+0000 7f2df5600700 1 -- 172.21.15.16:0/962872703 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563bb66f3d90 con 0x563bb67e5da0 2024-05-27T07:13:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.417+0000 7f2df5600700 10 client.26377 handle_command_reply: tid=0 2024-05-27T07:13:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.417+0000 7f2e1cd6db80 1 client.26377 shutdown 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.417+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 msgr2=0x563bb67d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.417+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2df80a8f70 tx=0x7f2df80a9040 comp rx=0 tx=0).stop 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 shutdown_connections 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb67e5da0 0x563bb67e8240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb67d24e0 0x563bb67d6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb6644e40 0x563bb67d5ca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2df5600700 0 client.26377 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/962872703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67d61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 >> 172.21.15.16:0/962872703 conn(0x563bb66a4fc0 msgr2=0x563bb67be580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.418+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 shutdown_connections 2024-05-27T07:13:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.420+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/962872703 wait complete. 2024-05-27T07:13:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.420+0000 7f2e1cd6db80 20 client.26377 trim_cache size 0 max 16384 2024-05-27T07:13:16.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-27T07:13:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.452+0000 7f2e1cd6db80 1 Processor -- start 2024-05-27T07:13:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.453+0000 7f2e1cd6db80 1 -- start start 2024-05-27T07:13:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de400f030 con 0x563bb67cc5c0 2024-05-27T07:13:16.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4010b70 con 0x563bb6644e40 2024-05-27T07:13:16.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2de400f700 con 0x563bb67d24e0 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.454+0000 7f2e07e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.455+0000 7f2e07400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.455+0000 7f2e07400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38388/0 (socket says 172.21.15.16:38388) 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.455+0000 7f2e07400700 1 -- 172.21.15.16:0/3160895992 learned_addr learned my addr 172.21.15.16:0/3160895992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.455+0000 7f2dfea00700 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.456+0000 7f2e07400700 1 -- 172.21.15.16:0/3160895992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 msgr2=0x563bb67d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.456+0000 7f2e07400700 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.456+0000 7f2e07400700 1 -- 172.21.15.16:0/3160895992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 msgr2=0x563bb67c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.456+0000 7f2e07400700 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.456+0000 7f2e07400700 1 -- 172.21.15.16:0/3160895992 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb67d8ea0 con 0x563bb6644e40 2024-05-27T07:13:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.457+0000 7f2e07400700 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2df80a64e0 tx=0x7f2df8006040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0efb121ad0d466 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.458+0000 7f2df7e00700 1 -- 172.21.15.16:0/3160895992 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a9970 con 0x563bb6644e40 2024-05-27T07:13:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.458+0000 7f2df7e00700 1 -- 172.21.15.16:0/3160895992 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2df8087020 con 0x563bb6644e40 2024-05-27T07:13:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.458+0000 7f2df7e00700 1 -- 172.21.15.16:0/3160895992 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80b1e50 con 0x563bb6644e40 2024-05-27T07:13:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.461+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/3160895992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 msgr2=0x563bb67e2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.461+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2df80a64e0 tx=0x7f2df8006040 comp rx=0 tx=0).stop 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/3160895992 shutdown_connections 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67e2190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/3160895992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/3160895992 >> 172.21.15.16:0/3160895992 conn(0x563bb66a4fc0 msgr2=0x563bb67c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/3160895992 shutdown_connections 2024-05-27T07:13:16.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.462+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/3160895992 wait complete. 2024-05-27T07:13:16.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.463+0000 7f2e1cd6db80 1 -- start start 2024-05-27T07:13:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.463+0000 7f2e1cd6db80 1 Processor -- start 2024-05-27T07:13:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.465+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.465+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.466+0000 7f2e1cd6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.466+0000 7f2e1cd6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de40104a0 con 0x563bb67cc5c0 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.466+0000 7f2e07400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.467+0000 7f2e07400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38396/0 (socket says 172.21.15.16:38396) 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.468+0000 7f2e07400700 1 -- 172.21.15.16:0/2466364848 learned_addr learned my addr 172.21.15.16:0/2466364848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.468+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2de400fdd0 con 0x563bb6644e40 2024-05-27T07:13:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.468+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2de400cff0 con 0x563bb67d24e0 2024-05-27T07:13:16.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.470+0000 7f2dfea00700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.470+0000 7f2e1cd6db80 10 client.0 _ll_register_callbacks cb 0x563bb668bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:16.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.470+0000 7f2e07e00700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.470+0000 7f2e07400700 1 -- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 msgr2=0x563bb67ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.471+0000 7f2e07400700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.471+0000 7f2e07400700 1 -- 172.21.15.16:0/2466364848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 msgr2=0x563bb67ed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.471+0000 7f2e07400700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67ed2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.472+0000 7f2e07400700 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb67edf80 con 0x563bb6644e40 2024-05-27T07:13:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.475+0000 7f2df5600700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-27T07:13:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.475+0000 7f2e07400700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007e90 tx=0x7f2df8009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879c117a9011138e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.476+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a9270 con 0x563bb6644e40 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.478+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df8087020 con 0x563bb6644e40 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.478+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f2df8088070 con 0x563bb6644e40 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.478+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2df80a9270 con 0x563bb6644e40 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.478+0000 7f2e1cd6db80 10 client.36446 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.478+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bb67edf80 con 0x563bb6644e40 2024-05-27T07:13:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.479+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2df8087020 con 0x563bb6644e40 2024-05-27T07:13:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.481+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2df8005cd0 con 0x563bb6644e40 2024-05-27T07:13:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 10 client.36446 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 10 client.36446 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:13:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 20 client.36446 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb6850a90 0x563bb6852eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 4 client.36446 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:13:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.483+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 --> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bb66ed570 con 0x563bb6850a90 2024-05-27T07:13:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.484+0000 7f2dfea00700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb6850a90 0x563bb6852eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.485+0000 7f2dfea00700 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb6850a90 0x563bb6852eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8d889099cba33a1 server_cookie=8297ef097f3f46bf in_seq=0 out_seq=0 2024-05-27T07:13:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.485+0000 7f2df5600700 10 client.36446 ms_handle_connect on v2:172.21.15.164:6838/3910557712 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.485+0000 7f2df5600700 1 -- 172.21.15.16:0/2466364848 <== mds.4 v2:172.21.15.164:6838/3910557712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bb66ed570 con 0x563bb6850a90 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.485+0000 7f2df5600700 10 client.36446 handle_command_reply: tid=0 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 client.36446 shutdown 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 msgr2=0x563bb67dea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007e90 tx=0x7f2df8009460 comp rx=0 tx=0).stop 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 shutdown_connections 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:6838/3910557712,v1:172.21.15.164:6839/3910557712] conn(0x563bb6850a90 0x563bb6852eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bb67d24e0 0x563bb67ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb6644e40 0x563bb67dea90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 --2- 172.21.15.16:0/2466364848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb67cc5c0 0x563bb67ed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.486+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 >> 172.21.15.16:0/2466364848 conn(0x563bb66a4fc0 msgr2=0x563bb67cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.487+0000 7f2df5600700 0 client.36446 ms_handle_reset on v2:172.21.15.164:6838/3910557712 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.487+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 shutdown_connections 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.487+0000 7f2e1cd6db80 1 -- 172.21.15.16:0/2466364848 wait complete. 2024-05-27T07:13:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.487+0000 7f2e1cd6db80 20 client.36446 trim_cache size 0 max 16384 2024-05-27T07:13:16.480 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:13:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 msgr2=0x7f2dec0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 0x7f2dec0b5000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009350 tx=0x7f2df000b540 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 msgr2=0x7f2e0819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e0819ce30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007ec0 tx=0x7f2df800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 shutdown_connections 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f2dec0b2af0 0x7f2dec0b5000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08109860 0x7f2e081a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e081045e0 0x7f2e0819ce30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 --2- 172.21.15.16:0/3867857399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e08108f10 0x7f2e0819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.489+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 >> 172.21.15.16:0/3867857399 conn(0x7f2e08100030 msgr2=0x7f2e0810fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.491+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 shutdown_connections 2024-05-27T07:13:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:16.491+0000 7f2e0e200700 1 -- 172.21.15.16:0/3867857399 wait complete. 2024-05-27T07:13:16.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T07:13:17.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:17 smithi016 ceph-mon[31415]: pgmap v388: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 4.0 MiB/s wr, 137 op/s 2024-05-27T07:13:17.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:17 smithi164 ceph-mon[35327]: pgmap v388: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 4.0 MiB/s wr, 137 op/s 2024-05-27T07:13:17.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:17 smithi069 ceph-mon[35276]: pgmap v388: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 4.0 MiB/s wr, 137 op/s 2024-05-27T07:13:17.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:13:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 -- 172.21.15.16:0/1141652487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c10a150 msgr2=0x7f926c103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 --2- 172.21.15.16:0/1141652487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c10a150 0x7f926c103d90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9258009900 tx=0x7f925802a410 comp rx=0 tx=0).stop 2024-05-27T07:13:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 -- 172.21.15.16:0/1141652487 shutdown_connections 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 --2- 172.21.15.16:0/1141652487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104400 0x7f926c1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 --2- 172.21.15.16:0/1141652487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c10a150 0x7f926c103d90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 --2- 172.21.15.16:0/1141652487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 -- 172.21.15.16:0/1141652487 >> 172.21.15.16:0/1141652487 conn(0x7f926c0ff480 msgr2=0x7f926c1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 -- 172.21.15.16:0/1141652487 shutdown_connections 2024-05-27T07:13:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.809+0000 7f9270c00640 1 -- 172.21.15.16:0/1141652487 wait complete. 2024-05-27T07:13:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.813+0000 7f9270c00640 1 Processor -- start 2024-05-27T07:13:17.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.815+0000 7f9270c00640 1 -- start start 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c104400 0x7f926c074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 0x7f926c06e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f926c113e20 con 0x7f926c104400 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f926c113ca0 con 0x7f926c109cb0 2024-05-27T07:13:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9270c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f926c113fa0 con 0x7f926c10a150 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f926a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 0x7f926c06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f926a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 0x7f926c06e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36840/0 (socket says 172.21.15.16:36840) 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f926a800640 1 -- 172.21.15.16:0/1494479983 learned_addr learned my addr 172.21.15.16:0/1494479983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.816+0000 7f9269e00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c104400 0x7f926c074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 -- 172.21.15.16:0/1494479983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 msgr2=0x7f926c06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 0x7f926c06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 -- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c104400 msgr2=0x7f926c074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c104400 0x7f926c074990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.817+0000 7f9269400640 1 -- 172.21.15.16:0/1494479983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c0724b0 con 0x7f926c109cb0 2024-05-27T07:13:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.819+0000 7f9269400640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c06dad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f925802aa00 tx=0x7f9258003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.820+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925803f070 con 0x7f926c109cb0 2024-05-27T07:13:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.820+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c072740 con 0x7f926c109cb0 2024-05-27T07:13:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.821+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9258003e50 con 0x7f926c109cb0 2024-05-27T07:13:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.821+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258035030 con 0x7f926c109cb0 2024-05-27T07:13:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.821+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c072c50 con 0x7f926c109cb0 2024-05-27T07:13:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.823+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c104f90 con 0x7f926c109cb0 2024-05-27T07:13:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.841+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9258042070 con 0x7f926c109cb0 2024-05-27T07:13:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.843+0000 7f9262a00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 0x7f92500b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.846+0000 7f9269e00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 0x7f92500b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.847+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f9258103190 con 0x7f926c109cb0 2024-05-27T07:13:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.848+0000 7f9269e00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 0x7f92500b4e60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f926c104990 tx=0x7f92640094f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:17.855+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92580ca730 con 0x7f926c109cb0 2024-05-27T07:13:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.346+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f926c072eb0 con 0x7f92500b2950 2024-05-27T07:13:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:18.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.461+0000 7f9262a00640 1 -- 172.21.15.16:0/1494479983 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f926c072eb0 con 0x7f92500b2950 2024-05-27T07:13:18.453 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:01:14", "first": "2024-05-27T07:02:00", "last": "2024-05-27T07:13:01", "last_pruned": "2024-05-27T07:13:01", "created_count": 12, "pruned_count": 6, "active": true} 2024-05-27T07:13:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.467+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 msgr2=0x7f92500b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.467+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 0x7f92500b4e60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f926c104990 tx=0x7f92640094f0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.467+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 msgr2=0x7f926c06dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.467+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c06dad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f925802aa00 tx=0x7f9258003c60 comp rx=0 tx=0).stop 2024-05-27T07:13:18.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 shutdown_connections 2024-05-27T07:13:18.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f92500b2950 0x7f92500b4e60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c10a150 0x7f926c06e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f926c109cb0 0x7f926c06dad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 --2- 172.21.15.16:0/1494479983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c104400 0x7f926c074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 >> 172.21.15.16:0/1494479983 conn(0x7f926c0ff480 msgr2=0x7f926c10f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:18.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 shutdown_connections 2024-05-27T07:13:18.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.469+0000 7f9270c00640 1 -- 172.21.15.16:0/1494479983 wait complete. 2024-05-27T07:13:18.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T07:13:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.953+0000 7fbc73a00640 1 -- 172.21.15.16:0/674005902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c112900 msgr2=0x7fbc6c118eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.953+0000 7fbc73a00640 1 --2- 172.21.15.16:0/674005902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c112900 0x7fbc6c118eb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009900 tx=0x7fbc6402a480 comp rx=0 tx=0).stop 2024-05-27T07:13:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.955+0000 7fbc73a00640 1 -- 172.21.15.16:0/674005902 shutdown_connections 2024-05-27T07:13:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.955+0000 7fbc73a00640 1 --2- 172.21.15.16:0/674005902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c112900 0x7fbc6c118eb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.955+0000 7fbc73a00640 1 --2- 172.21.15.16:0/674005902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.955+0000 7fbc73a00640 1 --2- 172.21.15.16:0/674005902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c10a490 0x7fbc6c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.955+0000 7fbc73a00640 1 -- 172.21.15.16:0/674005902 >> 172.21.15.16:0/674005902 conn(0x7fbc6c06e8f0 msgr2=0x7fbc6c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.956+0000 7fbc73a00640 1 -- 172.21.15.16:0/674005902 shutdown_connections 2024-05-27T07:13:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.956+0000 7fbc73a00640 1 -- 172.21.15.16:0/674005902 wait complete. 2024-05-27T07:13:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.957+0000 7fbc73a00640 1 Processor -- start 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.959+0000 7fbc73a00640 1 -- start start 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.959+0000 7fbc73a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c10a490 0x7fbc6c119670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.959+0000 7fbc73a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c119bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.959+0000 7fbc73a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.959+0000 7fbc73a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c11e6f0 con 0x7fbc6c10a490 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc73a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c11e570 con 0x7fbc6c111de0 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc73a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c11e870 con 0x7fbc6c112900 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc71a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc71000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c10a490 0x7fbc6c119670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc71a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36858/0 (socket says 172.21.15.16:36858) 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc71a00640 1 -- 172.21.15.16:0/447604439 learned_addr learned my addr 172.21.15.16:0/447604439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.960+0000 7fbc6be00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c119bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc6be00640 1 -- 172.21.15.16:0/447604439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 msgr2=0x7fbc6c11a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc6be00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc6be00640 1 -- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c10a490 msgr2=0x7fbc6c119670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc6be00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c10a490 0x7fbc6c119670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc6be00640 1 -- 172.21.15.16:0/447604439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c1a4f40 con 0x7fbc6c111de0 2024-05-27T07:13:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.961+0000 7fbc71a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:13:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.962+0000 7fbc6be00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c119bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cce0 tx=0x7fbc5c0075f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.963+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c004b20 con 0x7fbc6c111de0 2024-05-27T07:13:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.963+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc5c004cc0 con 0x7fbc6c111de0 2024-05-27T07:13:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.963+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1a5230 con 0x7fbc6c111de0 2024-05-27T07:13:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.963+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00f6b0 con 0x7fbc6c111de0 2024-05-27T07:13:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.964+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1a57d0 con 0x7fbc6c111de0 2024-05-27T07:13:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.965+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c10b510 con 0x7fbc6c111de0 2024-05-27T07:13:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.972+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbc5c005000 con 0x7fbc6c111de0 2024-05-27T07:13:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.975+0000 7fbc69600640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 0x7fbc4c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.975+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7fbc5c0e2d10 con 0x7fbc6c111de0 2024-05-27T07:13:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.976+0000 7fbc71000640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 0x7fbc4c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.977+0000 7fbc71000640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 0x7fbc4c0b4f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc540058c0 tx=0x7fbc54005c60 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:18.978+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc5c014030 con 0x7fbc6c111de0 2024-05-27T07:13:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.438+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 --> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fbc6c063100 con 0x7fbc4c0b2a20 2024-05-27T07:13:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.478+0000 7fbc69600640 1 -- 172.21.15.16:0/447604439 <== mgr.15297 v2:172.21.15.16:6800/1617637164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fbc6c063100 con 0x7fbc4c0b2a20 2024-05-27T07:13:19.471 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.483+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 msgr2=0x7fbc4c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.483+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 0x7fbc4c0b4f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc540058c0 tx=0x7fbc54005c60 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.483+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 msgr2=0x7fbc6c119bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.483+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c119bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cce0 tx=0x7fbc5c0075f0 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 shutdown_connections 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fbc4c0b2a20 0x7fbc4c0b4f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc6c112900 0x7fbc6c11a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc6c111de0 0x7fbc6c119bb0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cce0 tx=0x7fbc5c0075f0 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 --2- 172.21.15.16:0/447604439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c10a490 0x7fbc6c119670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 >> 172.21.15.16:0/447604439 conn(0x7fbc6c06e8f0 msgr2=0x7fbc6c0739c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.484+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 shutdown_connections 2024-05-27T07:13:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:19.485+0000 7fbc73a00640 1 -- 172.21.15.16:0/447604439 wait complete. 2024-05-27T07:13:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:19 smithi016 ceph-mon[31415]: pgmap v389: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-27T07:13:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:19 smithi016 ceph-mon[31415]: from='client.36449 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:19.517 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-27T07:13:19.531 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-27T07:13:19.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:19 smithi164 ceph-mon[35327]: pgmap v389: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-27T07:13:19.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:19 smithi164 ceph-mon[35327]: from='client.36449 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:19.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:19 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:13:19.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:19 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:19.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:19 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 5.153s CPU time. 2024-05-27T07:13:19.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:19 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:13:19.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:19 smithi069 ceph-mon[35276]: pgmap v389: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-05-27T07:13:19.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:19 smithi069 ceph-mon[35276]: from='client.36449 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:20.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:20 smithi016 ceph-mon[31415]: from='client.36452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:20.400 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:20 smithi164 podman[71442]: 2024-05-27 07:13:19.930256108 +0000 UTC m=+0.056921950 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:13:20.401 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:20 smithi164 ceph-mon[35327]: from='client.36452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:20.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:20 smithi069 ceph-mon[35276]: from='client.36452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:13:21.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.083+0000 7f55bd600700 1 -- 172.21.15.16:0/1667938698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b810e8e0 msgr2=0x7f55b8110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.083+0000 7f55bd600700 1 --2- 172.21.15.16:0/1667938698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b810e8e0 0x7f55b8110dc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f55b000b0a0 tx=0x7f55b000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 -- 172.21.15.16:0/1667938698 shutdown_connections 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 --2- 172.21.15.16:0/1667938698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b810e8e0 0x7f55b8110dc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 --2- 172.21.15.16:0/1667938698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b81054b0 0x7f55b81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 --2- 172.21.15.16:0/1667938698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 -- 172.21.15.16:0/1667938698 >> 172.21.15.16:0/1667938698 conn(0x7f55b80fedb0 msgr2=0x7f55b81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 -- 172.21.15.16:0/1667938698 shutdown_connections 2024-05-27T07:13:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.085+0000 7f55bd600700 1 -- 172.21.15.16:0/1667938698 wait complete. 2024-05-27T07:13:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.087+0000 7f55bd600700 1 Processor -- start 2024-05-27T07:13:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.087+0000 7f55bd600700 1 -- start start 2024-05-27T07:13:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.088+0000 7f55bd600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b81054b0 0x7f55b810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.089+0000 7f55bd600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.089+0000 7f55bd600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b810e8e0 0x7f55b81080c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.089+0000 7f55bd600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8112170 con 0x7f55b81054b0 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.089+0000 7f55b6000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.089+0000 7f55b6000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38442/0 (socket says 172.21.15.16:38442) 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.090+0000 7f55b6000700 1 -- 172.21.15.16:0/3386452345 learned_addr learned my addr 172.21.15.16:0/3386452345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.090+0000 7f55b7400700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b810e8e0 0x7f55b81080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.090+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8111ff0 con 0x7f55b8105e00 2024-05-27T07:13:21.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.090+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81122f0 con 0x7f55b810e8e0 2024-05-27T07:13:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.090+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b81054b0 0x7f55b810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.091+0000 7f55b6000700 1 -- 172.21.15.16:0/3386452345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b810e8e0 msgr2=0x7f55b81080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.091+0000 7f55b6000700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b810e8e0 0x7f55b81080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.091+0000 7f55b6000700 1 -- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b81054b0 msgr2=0x7f55b810de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.091+0000 7f55b6000700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b81054b0 0x7f55b810de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.091+0000 7f55b6000700 1 -- 172.21.15.16:0/3386452345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81088f0 con 0x7f55b8105e00 2024-05-27T07:13:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.092+0000 7f55b6000700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c920 tx=0x7f55a400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.093+0000 7f55af400700 1 -- 172.21.15.16:0/3386452345 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004a30 con 0x7f55b8105e00 2024-05-27T07:13:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.093+0000 7f55af400700 1 -- 172.21.15.16:0/3386452345 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55a40054b0 con 0x7f55b8105e00 2024-05-27T07:13:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.093+0000 7f55af400700 1 -- 172.21.15.16:0/3386452345 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4007be0 con 0x7f55b8105e00 2024-05-27T07:13:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.094+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b8108be0 con 0x7f55b8105e00 2024-05-27T07:13:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.094+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b80733d0 con 0x7f55b8105e00 2024-05-27T07:13:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.104+0000 7f55af400700 1 -- 172.21.15.16:0/3386452345 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55a4004c60 con 0x7f55b8105e00 2024-05-27T07:13:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.106+0000 7f55af400700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 0x7f559c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.107+0000 7f55af400700 1 -- 172.21.15.16:0/3386452345 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f55a40e27e0 con 0x7f55b8105e00 2024-05-27T07:13:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.108+0000 7f55cc1e3b80 1 Processor -- start 2024-05-27T07:13:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.108+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 0x7f559c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.108+0000 7f55cc1e3b80 1 -- start start 2024-05-27T07:13:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a03347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a032f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8a025ce40 con 0x55a8a0334d30 2024-05-27T07:13:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8a024c9e0 con 0x55a8a032edb0 2024-05-27T07:13:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.109+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8a0250a20 con 0x55a8a01afe40 2024-05-27T07:13:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.111+0000 7f55b7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a032f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.111+0000 7f55b6000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.112+0000 7f55b6000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50690/0 (socket says 172.21.15.16:50690) 2024-05-27T07:13:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.112+0000 7f55b6000700 1 -- 172.21.15.16:0/1996426691 learned_addr learned my addr 172.21.15.16:0/1996426691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6000700 1 -- 172.21.15.16:0/1996426691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 msgr2=0x55a8a03347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6000700 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a03347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6000700 1 -- 172.21.15.16:0/1996426691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 msgr2=0x55a8a032f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6000700 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a032f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6000700 1 -- 172.21.15.16:0/1996426691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0215420 con 0x55a8a0334d30 2024-05-27T07:13:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.113+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 0x7f559c0b5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005950 tx=0x7f55a800a770 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.114+0000 7f55b6000700 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f55a40a56a0 tx=0x7f55a40a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad605e636e2af24b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.114+0000 7f55a3e00700 1 -- 172.21.15.16:0/1996426691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40b2480 con 0x55a8a0334d30 2024-05-27T07:13:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.114+0000 7f55a3e00700 1 -- 172.21.15.16:0/1996426691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55a4007d80 con 0x55a8a0334d30 2024-05-27T07:13:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.114+0000 7f55a3e00700 1 -- 172.21.15.16:0/1996426691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40a9ae0 con 0x55a8a0334d30 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.116+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/1996426691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 msgr2=0x55a8a032e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.116+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f55a40a56a0 tx=0x7f55a40a9da0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/1996426691 shutdown_connections 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a032f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a03347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/1996426691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a032e870 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/1996426691 >> 172.21.15.16:0/1996426691 conn(0x55a8a020ffc0 msgr2=0x55a8a03298d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/1996426691 shutdown_connections 2024-05-27T07:13:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.117+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/1996426691 wait complete. 2024-05-27T07:13:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.118+0000 7f55cc1e3b80 1 -- start start 2024-05-27T07:13:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.118+0000 7f55cc1e3b80 1 Processor -- start 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 0x55a8a0338560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a0338fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8a0152fa0 con 0x55a8a0334d30 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8a01ed760 con 0x55a8a01afe40 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8a033b800 con 0x55a8a032edb0 2024-05-27T07:13:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.119+0000 7f55cc1e3b80 10 client.0 _ll_register_callbacks cb 0x55a8a01f6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.121+0000 7f55b6000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.121+0000 7f55b6000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36866/0 (socket says 172.21.15.16:36866) 2024-05-27T07:13:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.121+0000 7f55b6000700 1 -- 172.21.15.16:0/861671546 learned_addr learned my addr 172.21.15.16:0/861671546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.121+0000 7f55b6a00700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 0x55a8a0338560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.121+0000 7f55b7400700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a0338fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.122+0000 7f55b6000700 1 -- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 msgr2=0x55a8a0338560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.122+0000 7f55b6000700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 0x55a8a0338560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.122+0000 7f55b6000700 1 -- 172.21.15.16:0/861671546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 msgr2=0x55a8a0338fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.122+0000 7f55b6000700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a0338fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.122+0000 7f55b6000700 1 -- 172.21.15.16:0/861671546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a019c8d0 con 0x55a8a032edb0 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.123+0000 7f55b6a00700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 0x55a8a0338560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.123+0000 7f55b6000700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55a40a6890 tx=0x7f55a4005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2114158cd703292 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.123+0000 7f55a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.123+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40c7070 con 0x55a8a032edb0 2024-05-27T07:13:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.124+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f55a40a9a90 con 0x55a8a032edb0 2024-05-27T07:13:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.124+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a4007d80 con 0x55a8a032edb0 2024-05-27T07:13:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.124+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55a40b2480 con 0x55a8a032edb0 2024-05-27T07:13:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.124+0000 7f55cc1e3b80 10 client.26398 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.124+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a8a021d060 con 0x55a8a032edb0 2024-05-27T07:13:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55a4007d80 con 0x55a8a032edb0 2024-05-27T07:13:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f55a40ca360 con 0x55a8a032edb0 2024-05-27T07:13:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55cc1e3b80 10 client.26398 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55cc1e3b80 10 client.26398 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:13:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55cc1e3b80 20 client.26398 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.125+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a0348660 0x55a8a034aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.126+0000 7f55cc1e3b80 4 client.26398 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.126+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a025ed90 con 0x55a8a0348660 2024-05-27T07:13:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.126+0000 7f55b6a00700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a0348660 0x55a8a034aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.128+0000 7f55a0c00700 10 client.26398 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:13:21.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.128+0000 7f55b6a00700 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a0348660 0x55a8a034aa80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2dca7a9dd97ddec server_cookie=369bec5113007dba in_seq=0 out_seq=0 2024-05-27T07:13:21.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.129+0000 7f55a0c00700 1 -- 172.21.15.16:0/861671546 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8a025ed90 con 0x55a8a0348660 2024-05-27T07:13:21.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.129+0000 7f55a0c00700 10 client.26398 handle_command_reply: tid=0 2024-05-27T07:13:21.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.129+0000 7f55cc1e3b80 1 client.26398 shutdown 2024-05-27T07:13:21.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 msgr2=0x55a8a0338aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55a40a6890 tx=0x7f55a4005890 comp rx=0 tx=0).stop 2024-05-27T07:13:21.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 shutdown_connections 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a0338fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a0348660 0x55a8a034aa80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a01afe40 0x55a8a0338560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/861671546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a032edb0 0x55a8a0338aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 >> 172.21.15.16:0/861671546 conn(0x55a8a020ffc0 msgr2=0x55a8a0329580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.130+0000 7f55a0c00700 0 client.26398 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:13:21.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.131+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 shutdown_connections 2024-05-27T07:13:21.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.131+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/861671546 wait complete. 2024-05-27T07:13:21.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.132+0000 7f55cc1e3b80 20 client.26398 trim_cache size 0 max 16384 2024-05-27T07:13:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 Processor -- start 2024-05-27T07:13:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 -- start start 2024-05-27T07:13:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a032e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a032edb0 0x55a8a03374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 0x55a8a033aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c7d70 con 0x55a8a032edb0 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c98b0 con 0x55a8a0334d30 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.168+0000 7f55cc1e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c8440 con 0x55a8a01afe40 2024-05-27T07:13:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.169+0000 7f55b7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 0x55a8a033aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.169+0000 7f55b7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 0x55a8a033aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.16:38462/0 (socket says 172.21.15.16:38462) 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.169+0000 7f55b7400700 1 -- 172.21.15.16:0/3929817450 learned_addr learned my addr 172.21.15.16:0/3929817450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.169+0000 7f55b6000700 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a032edb0 0x55a8a03374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.169+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a032e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.170+0000 7f55b6a00700 1 -- 172.21.15.16:0/3929817450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 msgr2=0x55a8a033aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.170+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 0x55a8a033aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.170+0000 7f55b6a00700 1 -- 172.21.15.16:0/3929817450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a032edb0 msgr2=0x55a8a03374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.170+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a032edb0 0x55a8a03374c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.170+0000 7f55b6a00700 1 -- 172.21.15.16:0/3929817450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0336ad0 con 0x55a8a01afe40 2024-05-27T07:13:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.171+0000 7f55b6a00700 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a032e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a8a025ce40 tx=0x7f55a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedb20b04d159c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.171+0000 7f55a3400700 1 -- 172.21.15.16:0/3929817450 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8018070 con 0x55a8a01afe40 2024-05-27T07:13:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.172+0000 7f55a3400700 1 -- 172.21.15.16:0/3929817450 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a8a025ed90 con 0x55a8a01afe40 2024-05-27T07:13:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.172+0000 7f55a3400700 1 -- 172.21.15.16:0/3929817450 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a80053a0 con 0x55a8a01afe40 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/3929817450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 msgr2=0x55a8a032e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a032e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a8a025ce40 tx=0x7f55a8014000 comp rx=0 tx=0).stop 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/3929817450 shutdown_connections 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a0334d30 0x55a8a033aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a032e870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/3929817450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a032edb0 0x55a8a03374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/3929817450 >> 172.21.15.16:0/3929817450 conn(0x55a8a020ffc0 msgr2=0x55a8a0338520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/3929817450 shutdown_connections 2024-05-27T07:13:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.173+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/3929817450 wait complete. 2024-05-27T07:13:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.174+0000 7f55cc1e3b80 1 -- start start 2024-05-27T07:13:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.174+0000 7f55cc1e3b80 1 Processor -- start 2024-05-27T07:13:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.176+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.176+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a034abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.176+0000 7f55cc1e3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a034b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.176+0000 7f55b6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.176+0000 7f55b6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.16:36890/0 (socket says 172.21.15.16:36890) 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55b6a00700 1 -- 172.21.15.16:0/2546221171 learned_addr learned my addr 172.21.15.16:0/2546221171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55b6000700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a034abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c91e0 con 0x55a8a0334d30 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c8b10 con 0x55a8a032edb0 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81c5d10 con 0x55a8a01afe40 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55cc1e3b80 10 client.0 _ll_register_callbacks cb 0x55a8a01f6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55b7400700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a034b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55b6a00700 1 -- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 msgr2=0x55a8a034abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.177+0000 7f55b6a00700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a034abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55b6a00700 1 -- 172.21.15.16:0/2546221171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 msgr2=0x55a8a034b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55b6a00700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a034b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55b6a00700 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a034e2b0 con 0x55a8a01afe40 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55b6000700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a034abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:13:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55b6a00700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55a8012e90 tx=0x7f55a8012ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91e1f322caf4b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.178+0000 7f55a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.179+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004d70 con 0x55a8a01afe40 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.179+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f55a8004fb0 con 0x55a8a01afe40 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.179+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8a025ed90 con 0x55a8a01afe40 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.180+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55a80051b0 con 0x55a8a01afe40 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.180+0000 7f55cc1e3b80 10 client.26404 fetch_fsmap learned FSMap version 31 2024-05-27T07:13:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.180+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a8a03b1f50 con 0x55a8a01afe40 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a8a025ed90 con 0x55a8a01afe40 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f55a8038020 con 0x55a8a01afe40 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 10 client.26404 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 10 client.26404 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 20 client.26404 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a03b24a0 0x55a8a03b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 4 client.26404 mds_command: new command op to 34421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:13:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 --> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8a0258570 con 0x55a8a03b24a0 2024-05-27T07:13:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.183+0000 7f55b6000700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a03b24a0 0x55a8a03b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.184+0000 7f55a0c00700 10 client.26404 ms_handle_connect on v2:172.21.15.69:6832/123964468 2024-05-27T07:13:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.184+0000 7f55b6000700 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a03b24a0 0x55a8a03b48c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a4f921bf2c64563 server_cookie=e4d3b7627c0e26d4 in_seq=0 out_seq=0 2024-05-27T07:13:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.185+0000 7f55a0c00700 1 -- 172.21.15.16:0/2546221171 <== mds.3 v2:172.21.15.69:6832/123964468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8a0258570 con 0x55a8a03b24a0 2024-05-27T07:13:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.185+0000 7f55a0c00700 10 client.26404 handle_command_reply: tid=0 2024-05-27T07:13:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.185+0000 7f55cc1e3b80 1 client.26404 shutdown 2024-05-27T07:13:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.185+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 msgr2=0x55a8a034a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.185+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55a8012e90 tx=0x7f55a8012ec0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 shutdown_connections 2024-05-27T07:13:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8a0334d30 0x55a8a034b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8a01afe40 0x55a8a034a670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:6832/123964468,v1:172.21.15.69:6833/123964468] conn(0x55a8a03b24a0 0x55a8a03b48c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 --2- 172.21.15.16:0/2546221171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8a032edb0 0x55a8a034abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55a0c00700 0 client.26404 ms_handle_reset on v2:172.21.15.69:6832/123964468 2024-05-27T07:13:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.186+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 >> 172.21.15.16:0/2546221171 conn(0x55a8a020ffc0 msgr2=0x55a8a0338340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.192+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 shutdown_connections 2024-05-27T07:13:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.192+0000 7f55cc1e3b80 1 -- 172.21.15.16:0/2546221171 wait complete. 2024-05-27T07:13:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.192+0000 7f55cc1e3b80 20 client.26404 trim_cache size 0 max 16384 2024-05-27T07:13:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:13:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 msgr2=0x7f559c0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 0x7f559c0b5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005950 tx=0x7f55a800a770 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 msgr2=0x7f55b810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c920 tx=0x7f55a400cc30 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 shutdown_connections 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f559c0b2b30 0x7f559c0b5040 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005950 tx=0x7f55a800a770 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55b810e8e0 0x7f55b81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b81054b0 0x7f55b810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 --2- 172.21.15.16:0/3386452345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b8105e00 0x7f55b810e370 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.193+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 >> 172.21.15.16:0/3386452345 conn(0x7f55b80fedb0 msgr2=0x7f55b80ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.194+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 shutdown_connections 2024-05-27T07:13:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:21.196+0000 7f55bd600700 1 -- 172.21.15.16:0/3386452345 wait complete. 2024-05-27T07:13:21.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:21 smithi016 ceph-mon[31415]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-27T07:13:21.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:21 smithi164 ceph-mon[35327]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-27T07:13:21.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:21 smithi069 ceph-mon[35276]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-27T07:13:22.082 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:21 smithi164 podman[71442]: 2024-05-27 07:13:21.71312161 +0000 UTC m=+1.839787435 container create 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:13:22.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:13:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:23.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:23 smithi016 ceph-mon[31415]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 647 KiB/s wr, 168 op/s 2024-05-27T07:13:23.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:23 smithi069 ceph-mon[35276]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 647 KiB/s wr, 168 op/s 2024-05-27T07:13:24.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:24 smithi016 ceph-mon[31415]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:13:24.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:24 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:13:24.360 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe0b6600700 time 2024-05-27T07:13:24.089660+0000 2024-05-27T07:13:24.360 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:24.360 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.360 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe0cc1f4ba6] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.361 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.362 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.362 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: clone() 2024-05-27T07:13:24.362 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.105+0000 7fe0b6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe0b6600700 time 2024-05-27T07:13:24.089660+0000 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe0cc1f4ba6] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.363 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: clone() 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: *** Caught signal (Aborted) ** 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: in thread 7fe0b6600700 thread_name:md_log_replay 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.364 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe0cad71d20] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: gsignal() 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: abort() 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe0cc1f4ced] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.365 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 19: clone() 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.111+0000 7fe0b6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:24.366 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: in thread 7fe0b6600700 thread_name:md_log_replay 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe0cad71d20] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: gsignal() 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: abort() 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe0cc1f4ced] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.367 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.368 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 19: clone() 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: -1> 2024-05-27T07:13:24.105+0000 7fe0b6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe0b6600700 time 2024-05-27T07:13:24.089660+0000 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe0cc1f4ba6] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.369 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: clone() 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 0> 2024-05-27T07:13:24.111+0000 7fe0b6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: in thread 7fe0b6600700 thread_name:md_log_replay 2024-05-27T07:13:24.370 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe0cad71d20] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: gsignal() 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: abort() 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe0cc1f4ced] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.371 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.372 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 19: clone() 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: -1> 2024-05-27T07:13:24.105+0000 7fe0b6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe0b6600700 time 2024-05-27T07:13:24.089660+0000 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe0cc1f4ba6] 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.373 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: clone() 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.374 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 0> 2024-05-27T07:13:24.111+0000 7fe0b6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: in thread 7fe0b6600700 thread_name:md_log_replay 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe0cad71d20] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2: gsignal() 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 3: abort() 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe0cc1f4ced] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe0cc1f4dd1] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f9e15ae23d] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f9e1599971] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f9e1597fa0] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f9e1598651] 2024-05-27T07:13:24.375 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f9e152d347] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f9e122987e] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f9e122ee94] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f9e1239969] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f9e15d248c] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f9e15db9cb] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 16: (Thread::entry_wrapper()+0x43) [0x7fe0cc1c372d] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe0cc1c3749] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe0cad671ca] 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 19: clone() 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[70345]: 2024-05-27T07:13:24.376 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:24 smithi016 podman[111560]: 2024-05-27 07:13:24.306542853 +0000 UTC m=+0.074957701 container died 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-05-27T07:13:24.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:24 smithi069 ceph-mon[35276]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:13:24.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:24 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:13:24.767 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:23 smithi164 ceph-mon[35327]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 647 KiB/s wr, 168 op/s 2024-05-27T07:13:24.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:24 smithi164 ceph-mon[35327]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:13:24.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:24 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:13:25.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 podman[71442]: 2024-05-27 07:13:25.139704852 +0000 UTC m=+5.266370709 container init 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:13:25.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 podman[71442]: 2024-05-27 07:13:25.150056727 +0000 UTC m=+5.276722561 container start 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T07:13:25.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 bash[71442]: 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 2024-05-27T07:13:25.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 ceph-mds[71462]: -- 172.21.15.164:0/165454754 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa24c79a40 con 0x55fa24cc1a80 2024-05-27T07:13:25.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: starting mds.c at 2024-05-27T07:13:25.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:25 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:25.592 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:25 smithi016 podman[111560]: 2024-05-27 07:13:25.215168132 +0000 UTC m=+0.983582982 container remove 3af778ea9a2a2a27be7f70aa6120d6e7751ecd83d6b703b99dbe3771ad54abfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:13:25.592 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:25 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:13:26.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:25 smithi016 ceph-mon[31415]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 KiB/s wr, 162 op/s 2024-05-27T07:13:26.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:25 smithi069 ceph-mon[35276]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 KiB/s wr, 162 op/s 2024-05-27T07:13:26.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:25 smithi164 ceph-mon[35327]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 KiB/s wr, 162 op/s 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 -- 172.21.15.16:0/3085807387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 msgr2=0x7f2974109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 --2- 172.21.15.16:0/3085807387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f2974109130 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f29600099c0 tx=0x7f2960009cd0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 -- 172.21.15.16:0/3085807387 shutdown_connections 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 --2- 172.21.15.16:0/3085807387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2974109670 0x7f2974110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 --2- 172.21.15.16:0/3085807387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29741043f0 0x7f2974104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 --2- 172.21.15.16:0/3085807387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f2974109130 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 -- 172.21.15.16:0/3085807387 >> 172.21.15.16:0/3085807387 conn(0x7f2974079d00 msgr2=0x7f297407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 -- 172.21.15.16:0/3085807387 shutdown_connections 2024-05-27T07:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.633+0000 7f2973200700 1 -- 172.21.15.16:0/3085807387 wait complete. 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.634+0000 7f2973200700 1 Processor -- start 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.634+0000 7f2973200700 1 -- start start 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29741043f0 0x7f297419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2974109670 0x7f29741a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2974114180 con 0x7f2974108d20 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2974114000 con 0x7f29741043f0 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2973200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2974114300 con 0x7f2974109670 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2971400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2971400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45970/0 (socket says 172.21.15.16:45970) 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2971400700 1 -- 172.21.15.16:0/3428911480 learned_addr learned my addr 172.21.15.16:0/3428911480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2972800700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2974109670 0x7f29741a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.635+0000 7f2971e00700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29741043f0 0x7f297419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 -- 172.21.15.16:0/3428911480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2974109670 msgr2=0x7f29741a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2974109670 0x7f29741a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 -- 172.21.15.16:0/3428911480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29741043f0 msgr2=0x7f297419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29741043f0 0x7f297419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 -- 172.21.15.16:0/3428911480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29741a1c60 con 0x7f2974108d20 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.636+0000 7f2971400700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f296000b580 tx=0x7f2960004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.637+0000 7f296aa00700 1 -- 172.21.15.16:0/3428911480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960009e60 con 0x7f2974108d20 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.637+0000 7f296aa00700 1 -- 172.21.15.16:0/3428911480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2960004830 con 0x7f2974108d20 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.637+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741a1ef0 con 0x7f2974108d20 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.637+0000 7f296aa00700 1 -- 172.21.15.16:0/3428911480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960005280 con 0x7f2974108d20 2024-05-27T07:13:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.637+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a23d0 con 0x7f2974108d20 2024-05-27T07:13:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.641+0000 7f296aa00700 1 -- 172.21.15.16:0/3428911480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2960005420 con 0x7f2974108d20 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.642+0000 7f298763eb80 1 Processor -- start 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 -- start start 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd2835ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f296aa00700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 0x7f29580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2843020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555dd2764e40 con 0x555dd2836410 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555dd27549e0 con 0x555dd26b7e40 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.643+0000 7f298763eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555dd2758a20 con 0x555dd2836820 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f296aa00700 1 -- 172.21.15.16:0/3428911480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f29600e3b60 con 0x7f2974108d20 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f2971e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f2971e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45978/0 (socket says 172.21.15.16:45978) 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f2971e00700 1 -- 172.21.15.16:0/2116923447 learned_addr learned my addr 172.21.15.16:0/2116923447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f2972800700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd2835ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.644+0000 7f2972800700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2843020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971400700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 0x7f29580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 -- 172.21.15.16:0/2116923447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 msgr2=0x555dd2843020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2843020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 -- 172.21.15.16:0/2116923447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 msgr2=0x555dd2835ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd2835ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 -- 172.21.15.16:0/2116923447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd271d420 con 0x555dd2836410 2024-05-27T07:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2972800700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2843020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971e00700 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d10 tx=0x7f29640116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316721448dd66146 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.645+0000 7f2971400700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 0x7f29580b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f296000c010 tx=0x7f2960003f70 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.646+0000 7f295f400700 1 -- 172.21.15.16:0/2116923447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964009e60 con 0x555dd2836410 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.646+0000 7f295f400700 1 -- 172.21.15.16:0/2116923447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2964011e20 con 0x555dd2836410 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.646+0000 7f295f400700 1 -- 172.21.15.16:0/2116923447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400f7e0 con 0x555dd2836410 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 -- 172.21.15.16:0/2116923447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 msgr2=0x555dd2840ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d10 tx=0x7f29640116c0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 -- 172.21.15.16:0/2116923447 shutdown_connections 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2843020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd2835ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 --2- 172.21.15.16:0/2116923447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840ad0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 -- 172.21.15.16:0/2116923447 >> 172.21.15.16:0/2116923447 conn(0x555dd2717fc0 msgr2=0x555dd2831d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 -- 172.21.15.16:0/2116923447 shutdown_connections 2024-05-27T07:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.647+0000 7f298763eb80 1 -- 172.21.15.16:0/2116923447 wait complete. 2024-05-27T07:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 -- start start 2024-05-27T07:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 Processor -- start 2024-05-27T07:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd28404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2840f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555dd265afa0 con 0x555dd2836410 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555dd26f5760 con 0x555dd26b7e40 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.648+0000 7f298763eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555dd2843990 con 0x555dd2836820 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f298763eb80 10 client.0 _ll_register_callbacks cb 0x555dd26fec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45980/0 (socket says 172.21.15.16:45980) 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 -- 172.21.15.16:0/1404842652 learned_addr learned my addr 172.21.15.16:0/1404842652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2972800700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd28404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2972800700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2840f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 -- 172.21.15.16:0/1404842652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 msgr2=0x555dd2840f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2840f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 -- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 msgr2=0x555dd28404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd28404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.649+0000 7f2971e00700 1 -- 172.21.15.16:0/1404842652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd26a48d0 con 0x555dd2836410 2024-05-27T07:13:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.650+0000 7f2971e00700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2964000c00 tx=0x7f2964012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e233a525d073a780 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.650+0000 7f2953e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.650+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964027020 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.650+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f2964012ab0 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.650+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2964011e20 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2964032050 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 10 client.16539 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dd2725060 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2964011e20 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f29640047c0 con 0x555dd2836410 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 10 client.16539 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 10 client.16539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 20 client.16539 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd2850860 0x555dd2852c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 4 client.16539 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.651+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dd2766d90 con 0x555dd2850860 2024-05-27T07:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.652+0000 7f2972800700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd2850860 0x555dd2852c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.653+0000 7f2972800700 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd2850860 0x555dd2852c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2efd8963529854c3 server_cookie=d9ea7a82fa7be2e2 in_seq=0 out_seq=0 2024-05-27T07:13:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.653+0000 7f2953e00700 10 client.16539 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.655+0000 7f2953e00700 1 -- 172.21.15.16:0/1404842652 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555dd2766d90 con 0x555dd2850860 2024-05-27T07:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.655+0000 7f2953e00700 10 client.16539 handle_command_reply: tid=0 2024-05-27T07:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 client.16539 shutdown 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 msgr2=0x555dd2840a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2964000c00 tx=0x7f2964012680 comp rx=0 tx=0).stop 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 shutdown_connections 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd2850860 0x555dd2852c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836820 0x555dd2840f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd26b7e40 0x555dd28404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 --2- 172.21.15.16:0/1404842652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836410 0x555dd2840a20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 >> 172.21.15.16:0/1404842652 conn(0x555dd2717fc0 msgr2=0x555dd2830ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f2953e00700 0 client.16539 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 shutdown_connections 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 1 -- 172.21.15.16:0/1404842652 wait complete. 2024-05-27T07:13:26.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.656+0000 7f298763eb80 20 client.16539 trim_cache size 0 max 16384 2024-05-27T07:13:26.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.673+0000 7f298763eb80 1 Processor -- start 2024-05-27T07:13:26.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 -- start start 2024-05-27T07:13:26.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836410 0x555dd2842980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836820 0x555dd284ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b2690 con 0x555dd26b7e40 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b41d0 con 0x555dd2836820 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f298763eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b2d60 con 0x555dd2836410 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f2972800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f2972800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45986/0 (socket says 172.21.15.16:45986) 2024-05-27T07:13:26.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.674+0000 7f2972800700 1 -- 172.21.15.16:0/3024833129 learned_addr learned my addr 172.21.15.16:0/3024833129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2971e00700 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836410 0x555dd2842980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 -- 172.21.15.16:0/3024833129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836410 msgr2=0x555dd2842980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836410 0x555dd2842980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 -- 172.21.15.16:0/3024833129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836820 msgr2=0x555dd284ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836820 0x555dd284ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 -- 172.21.15.16:0/3024833129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd28434c0 con 0x555dd26b7e40 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f2972800700 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f296c009ee0 tx=0x7f296c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb7e5beb1bc9a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f295ea00700 1 -- 172.21.15.16:0/3024833129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004bb0 con 0x555dd26b7e40 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f295ea00700 1 -- 172.21.15.16:0/3024833129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c025030 con 0x555dd26b7e40 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.675+0000 7f295ea00700 1 -- 172.21.15.16:0/3024833129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0051a0 con 0x555dd26b7e40 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 -- 172.21.15.16:0/3024833129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 msgr2=0x555dd2831b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f296c009ee0 tx=0x7f296c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 -- 172.21.15.16:0/3024833129 shutdown_connections 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836820 0x555dd284ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd26b7e40 0x555dd2831b30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 --2- 172.21.15.16:0/3024833129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd2836410 0x555dd2842980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.676+0000 7f298763eb80 1 -- 172.21.15.16:0/3024833129 >> 172.21.15.16:0/3024833129 conn(0x555dd2717fc0 msgr2=0x555dd28404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.677+0000 7f298763eb80 1 -- 172.21.15.16:0/3024833129 shutdown_connections 2024-05-27T07:13:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.677+0000 7f298763eb80 1 -- 172.21.15.16:0/3024833129 wait complete. 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.677+0000 7f298763eb80 1 -- start start 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.677+0000 7f298763eb80 1 Processor -- start 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd26b7e40 0x555dd2843ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836410 0x555dd2844220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b3b00 con 0x555dd2836820 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b3430 con 0x555dd2836410 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b0650 con 0x555dd26b7e40 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f298763eb80 10 client.0 _ll_register_callbacks cb 0x555dd26fec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f2972800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f2972800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45996/0 (socket says 172.21.15.16:45996) 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.678+0000 7f2972800700 1 -- 172.21.15.16:0/3097670676 learned_addr learned my addr 172.21.15.16:0/3097670676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd26b7e40 0x555dd2843ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2971e00700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836410 0x555dd2844220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 -- 172.21.15.16:0/3097670676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd26b7e40 msgr2=0x555dd2843ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd26b7e40 0x555dd2843ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 -- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836410 msgr2=0x555dd2844220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836410 0x555dd2844220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.679+0000 7f2972800700 1 -- 172.21.15.16:0/3097670676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd28371d0 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f2972800700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f296c009cf0 tx=0x7f296c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f8e9a21184d6b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f2953e00700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c01b870 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9613+0+0 (secure 0 0 0) 0x7f296c01ba10 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c025030 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f296c03e050 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f298763eb80 10 client.16545 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.680+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dd28ba490 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c025030 con 0x555dd2836820 2024-05-27T07:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f296c036020 con 0x555dd2836820 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 10 client.16545 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 20 client.16545 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd28bab10 0x555dd28bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 4 client.16545 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.681+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 --> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dd2760570 con 0x555dd28bab10 2024-05-27T07:13:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.682+0000 7f2971e00700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd28bab10 0x555dd28bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f2971e00700 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd28bab10 0x555dd28bcf30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=378910c53134a5db server_cookie=7df5ccab30cab586 in_seq=0 out_seq=0 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f2953e00700 10 client.16545 ms_handle_connect on v2:172.21.15.69:6834/4223174404 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f2953e00700 1 -- 172.21.15.16:0/3097670676 <== mds.2 v2:172.21.15.69:6834/4223174404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dd2760570 con 0x555dd28bab10 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f2953e00700 10 client.16545 handle_command_reply: tid=0 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 client.16545 shutdown 2024-05-27T07:13:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 msgr2=0x555dd28b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f296c009cf0 tx=0x7f296c00b040 comp rx=0 tx=0).stop 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 shutdown_connections 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:6834/4223174404,v1:172.21.15.69:6835/4223174404] conn(0x555dd28bab10 0x555dd28bcf30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dd2836820 0x555dd28b90e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.683+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555dd26b7e40 0x555dd2843ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f298763eb80 1 --2- 172.21.15.16:0/3097670676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dd2836410 0x555dd2844220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 >> 172.21.15.16:0/3097670676 conn(0x555dd2717fc0 msgr2=0x555dd2840ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f2953e00700 0 client.16545 ms_handle_reset on v2:172.21.15.69:6834/4223174404 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 shutdown_connections 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f298763eb80 1 -- 172.21.15.16:0/3097670676 wait complete. 2024-05-27T07:13:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.684+0000 7f298763eb80 20 client.16545 trim_cache size 0 max 16384 2024-05-27T07:13:26.676 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:13:26.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 msgr2=0x7f29580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 0x7f29580b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f296000c010 tx=0x7f2960003f70 comp rx=0 tx=0).stop 2024-05-27T07:13:26.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 msgr2=0x7f297419d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f296000b580 tx=0x7f2960004690 comp rx=0 tx=0).stop 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 shutdown_connections 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f29580b29f0 0x7f29580b4f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2974109670 0x7f29741a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29741043f0 0x7f297419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 --2- 172.21.15.16:0/3428911480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974108d20 0x7f297419d330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 >> 172.21.15.16:0/3428911480 conn(0x7f2974079d00 msgr2=0x7f2974110df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 shutdown_connections 2024-05-27T07:13:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:26.685+0000 7f2973200700 1 -- 172.21.15.16:0/3428911480 wait complete. 2024-05-27T07:13:26.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:26 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:13:26.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:26 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service: Consumed 2.472s CPU time. 2024-05-27T07:13:27.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:26 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/2467065171,v1:172.21.15.164:6837/2467065171] up:boot 2024-05-27T07:13:27.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:26 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:27.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:26 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:27.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:26 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/2467065171,v1:172.21.15.164:6837/2467065171] up:boot 2024-05-27T07:13:27.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:26 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:27.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:26 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:27.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:26 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/2467065171,v1:172.21.15.164:6837/2467065171] up:boot 2024-05-27T07:13:27.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:26 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:27.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:26 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:27.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:13:28.189 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:28.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:27 smithi164 ceph-mon[35327]: pgmap v393: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 14 KiB/s wr, 202 op/s 2024-05-27T07:13:28.347 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:27 smithi016 ceph-mon[31415]: pgmap v393: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 14 KiB/s wr, 202 op/s 2024-05-27T07:13:28.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:27 smithi069 ceph-mon[35276]: pgmap v393: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 14 KiB/s wr, 202 op/s 2024-05-27T07:13:30.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:29 smithi016 ceph-mon[31415]: pgmap v394: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 4.8 KiB/s wr, 182 op/s 2024-05-27T07:13:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:29 smithi016 ceph-mon[31415]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:13:30.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:29 smithi069 ceph-mon[35276]: pgmap v394: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 4.8 KiB/s wr, 182 op/s 2024-05-27T07:13:30.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:29 smithi069 ceph-mon[35276]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:13:30.134 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:29 smithi164 ceph-mon[35327]: pgmap v394: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 4.8 KiB/s wr, 182 op/s 2024-05-27T07:13:30.134 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:29 smithi164 ceph-mon[35327]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:13:31.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:31 smithi016 ceph-mon[31415]: pgmap v396: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 5.7 KiB/s wr, 180 op/s 2024-05-27T07:13:32.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:31 smithi069 ceph-mon[35276]: pgmap v396: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 5.7 KiB/s wr, 180 op/s 2024-05-27T07:13:32.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:31 smithi164 ceph-mon[35327]: pgmap v396: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 5.7 KiB/s wr, 180 op/s 2024-05-27T07:13:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 -- 172.21.15.16:0/2877123889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde181045c0 msgr2=0x7fde181049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 --2- 172.21.15.16:0/2877123889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde181045c0 0x7fde181049d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0099c0 tx=0x7fde0c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 -- 172.21.15.16:0/2877123889 shutdown_connections 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 --2- 172.21.15.16:0/2877123889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde18109840 0x7fde181111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 --2- 172.21.15.16:0/2877123889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde181045c0 0x7fde181049d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 --2- 172.21.15.16:0/2877123889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 0x7fde18109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 -- 172.21.15.16:0/2877123889 >> 172.21.15.16:0/2877123889 conn(0x7fde180ffe90 msgr2=0x7fde181022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 -- 172.21.15.16:0/2877123889 shutdown_connections 2024-05-27T07:13:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.483+0000 7fde20600700 1 -- 172.21.15.16:0/2877123889 wait complete. 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.484+0000 7fde20600700 1 Processor -- start 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 -- start start 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 0x7fde1819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 0x7fde1819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde18114370 con 0x7fde18109840 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde181141f0 con 0x7fde18108ef0 2024-05-27T07:13:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.485+0000 7fde20600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde181144f0 con 0x7fde181045c0 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41174/0 (socket says 172.21.15.16:41174) 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 -- 172.21.15.16:0/93583139 learned_addr learned my addr 172.21.15.16:0/93583139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1d200700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 0x7fde1819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1dc00700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 0x7fde1819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 -- 172.21.15.16:0/93583139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 msgr2=0x7fde1819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 0x7fde1819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 -- 172.21.15.16:0/93583139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 msgr2=0x7fde1819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 0x7fde1819d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1e600700 1 -- 172.21.15.16:0/93583139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde181a1e10 con 0x7fde18109840 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.486+0000 7fde1dc00700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 0x7fde1819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde1e600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde10007ee0 tx=0x7fde1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde16000700 1 -- 172.21.15.16:0/93583139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000dbd0 con 0x7fde18109840 2024-05-27T07:13:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde16000700 1 -- 172.21.15.16:0/93583139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde1000dd70 con 0x7fde18109840 2024-05-27T07:13:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde181a2100 con 0x7fde18109840 2024-05-27T07:13:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde16000700 1 -- 172.21.15.16:0/93583139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100058d0 con 0x7fde18109840 2024-05-27T07:13:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.487+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde181a9a20 con 0x7fde18109840 2024-05-27T07:13:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.491+0000 7fde16000700 1 -- 172.21.15.16:0/93583139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fde1001a020 con 0x7fde18109840 2024-05-27T07:13:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.492+0000 7fde2f1bdb80 1 Processor -- start 2024-05-27T07:13:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.492+0000 7fde2f1bdb80 1 -- start start 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.492+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f40870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f788e6ee40 con 0x55f788ead9f0 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f788e5e9e0 con 0x55f788f40e70 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f788e62a20 con 0x55f788dc1e40 2024-05-27T07:13:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde16000700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 0x7fde000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41188/0 (socket says 172.21.15.16:41188) 2024-05-27T07:13:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde1d200700 1 -- 172.21.15.16:0/1626282402 learned_addr learned my addr 172.21.15.16:0/1626282402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde1dc00700 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f40870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.493+0000 7fde16000700 1 -- 172.21.15.16:0/93583139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fde100e21b0 con 0x7fde18109840 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1e600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 0x7fde000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 -- 172.21.15.16:0/1626282402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 msgr2=0x55f788f40870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f40870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 -- 172.21.15.16:0/1626282402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 msgr2=0x55f788f4d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 -- 172.21.15.16:0/1626282402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f788e27420 con 0x55f788ead9f0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1d200700 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fde04012650 tx=0x7fde0400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93309591e89754ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde1e600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 0x7fde000b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde10004fd0 tx=0x7fde10009390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.494+0000 7fde0aa00700 1 -- 172.21.15.16:0/1626282402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04004030 con 0x55f788ead9f0 2024-05-27T07:13:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.495+0000 7fde0aa00700 1 -- 172.21.15.16:0/1626282402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde04004830 con 0x55f788ead9f0 2024-05-27T07:13:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.495+0000 7fde0aa00700 1 -- 172.21.15.16:0/1626282402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04005220 con 0x55f788ead9f0 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.495+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/1626282402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 msgr2=0x55f788f40330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.495+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fde04012650 tx=0x7fde0400b8c0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/1626282402 shutdown_connections 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f40330 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/1626282402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f40870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/1626282402 >> 172.21.15.16:0/1626282402 conn(0x55f788e5b390 msgr2=0x55f788f3c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/1626282402 shutdown_connections 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/1626282402 wait complete. 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.496+0000 7fde2f1bdb80 1 -- start start 2024-05-27T07:13:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 Processor -- start 2024-05-27T07:13:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f788d64fa0 con 0x55f788ead9f0 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f788dff760 con 0x55f788f40e70 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.497+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f788f4db80 con 0x55f788dc1e40 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde2f1bdb80 10 client.0 _ll_register_callbacks cb 0x55f788e08c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41202/0 (socket says 172.21.15.16:41202) 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 -- 172.21.15.16:0/3934430367 learned_addr learned my addr 172.21.15.16:0/3934430367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1d200700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1d200700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 -- 172.21.15.16:0/3934430367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 msgr2=0x55f788f4a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f4a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 -- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 msgr2=0x55f788f4b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.498+0000 7fde1dc00700 1 -- 172.21.15.16:0/3934430367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f788dae8d0 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.499+0000 7fde1dc00700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fde0c015000 tx=0x7fde0c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ffe7c283943f3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.499+0000 7fddff400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.499+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c028060 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.499+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fde0c00bc20 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.499+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde04004830 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde0c02f050 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 10 client.16557 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f788e2f060 con 0x55f788ead9f0 2024-05-27T07:13:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde04004830 con 0x55f788ead9f0 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fde0c00f6f0 con 0x55f788ead9f0 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 10 client.16557 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 10 client.16557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 20 client.16557 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f5ab60 0x55f788f5cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 4 client.16557 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.500+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f788e70d90 con 0x55f788f5ab60 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.501+0000 7fde1d200700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f5ab60 0x55f788f5cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.502+0000 7fde1d200700 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f5ab60 0x55f788f5cf80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15c564224d4cf175 server_cookie=59c5b0691c6c5dae in_seq=0 out_seq=0 2024-05-27T07:13:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.502+0000 7fddff400700 10 client.16557 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:13:32.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.504+0000 7fddff400700 1 -- 172.21.15.16:0/3934430367 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f788e70d90 con 0x55f788f5ab60 2024-05-27T07:13:32.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.504+0000 7fddff400700 10 client.16557 handle_command_reply: tid=0 2024-05-27T07:13:32.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.504+0000 7fde2f1bdb80 1 client.16557 shutdown 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 msgr2=0x55f788f4abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fde0c015000 tx=0x7fde0c0057e0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 shutdown_connections 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f5ab60 0x55f788f5cf80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788f40e70 0x55f788f4b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788dc1e40 0x55f788f4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/3934430367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788ead9f0 0x55f788f4abc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 >> 172.21.15.16:0/3934430367 conn(0x55f788e5b390 msgr2=0x55f788f403f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fddff400700 0 client.16557 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:13:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 shutdown_connections 2024-05-27T07:13:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/3934430367 wait complete. 2024-05-27T07:13:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.505+0000 7fde2f1bdb80 20 client.16557 trim_cache size 0 max 16384 2024-05-27T07:13:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 Processor -- start 2024-05-27T07:13:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 -- start start 2024-05-27T07:13:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b2780 con 0x55f788dc1e40 2024-05-27T07:13:32.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b42c0 con 0x55f788ead9f0 2024-05-27T07:13:32.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.522+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b2e50 con 0x55f788f40e70 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41212/0 (socket says 172.21.15.16:41212) 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 -- 172.21.15.16:0/51389783 learned_addr learned my addr 172.21.15.16:0/51389783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1dc00700 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 -- 172.21.15.16:0/51389783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 msgr2=0x55f788f56c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 -- 172.21.15.16:0/51389783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 msgr2=0x55f788f40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 -- 172.21.15.16:0/51389783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f788f4b740 con 0x55f788dc1e40 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.523+0000 7fde1d200700 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde0400bd30 tx=0x7fde0400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1261b01b50adaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.524+0000 7fde0a000700 1 -- 172.21.15.16:0/51389783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04028070 con 0x55f788dc1e40 2024-05-27T07:13:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.524+0000 7fde0a000700 1 -- 172.21.15.16:0/51389783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde04004830 con 0x55f788dc1e40 2024-05-27T07:13:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.524+0000 7fde0a000700 1 -- 172.21.15.16:0/51389783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04010960 con 0x55f788dc1e40 2024-05-27T07:13:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/51389783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 msgr2=0x55f788f45e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde0400bd30 tx=0x7fde0400b8c0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/51389783 shutdown_connections 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f56c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f45e90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/51389783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/51389783 >> 172.21.15.16:0/51389783 conn(0x55f788e5b390 msgr2=0x55f788f40120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/51389783 shutdown_connections 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/51389783 wait complete. 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.525+0000 7fde2f1bdb80 1 -- start start 2024-05-27T07:13:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 Processor -- start 2024-05-27T07:13:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f44a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b3bf0 con 0x55f788dc1e40 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b3520 con 0x55f788ead9f0 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde181b0740 con 0x55f788f40e70 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde2f1bdb80 10 client.0 _ll_register_callbacks cb 0x55f788e08c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde1d200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41214/0 (socket says 172.21.15.16:41214) 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.526+0000 7fde1d200700 1 -- 172.21.15.16:0/4263172992 learned_addr learned my addr 172.21.15.16:0/4263172992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1dc00700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f44a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 -- 172.21.15.16:0/4263172992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 msgr2=0x55f788f44f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f44f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 -- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 msgr2=0x55f788f44a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f44a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 -- 172.21.15.16:0/4263172992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f788f51f90 con 0x55f788dc1e40 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1dc00700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fde1d200700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde04010580 tx=0x7fde0400b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d184a3cf72a9a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.527+0000 7fddff400700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04028040 con 0x55f788dc1e40 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde04004830 con 0x55f788dc1e40 2024-05-27T07:13:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fde040105e0 con 0x55f788dc1e40 2024-05-27T07:13:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde04010800 con 0x55f788dc1e40 2024-05-27T07:13:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 10 client.16563 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f788f531f0 con 0x55f788dc1e40 2024-05-27T07:13:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde04004830 con 0x55f788dc1e40 2024-05-27T07:13:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fde0401e3c0 con 0x55f788dc1e40 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.528+0000 7fde2f1bdb80 20 client.16563 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.529+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f53670 0x55f788fc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.529+0000 7fde2f1bdb80 4 client.16563 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.529+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 --> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f788e6a570 con 0x55f788f53670 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.529+0000 7fde1dc00700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f53670 0x55f788fc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.530+0000 7fde1dc00700 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f53670 0x55f788fc7640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b57b55cd12917b9a server_cookie=1bc85b4c0ea2a61e in_seq=0 out_seq=0 2024-05-27T07:13:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.530+0000 7fddff400700 10 client.16563 ms_handle_connect on v2:172.21.15.69:6836/2624187710 2024-05-27T07:13:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fddff400700 1 -- 172.21.15.16:0/4263172992 <== mds.1 v2:172.21.15.69:6836/2624187710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f788e6a570 con 0x55f788f53670 2024-05-27T07:13:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fddff400700 10 client.16563 handle_command_reply: tid=0 2024-05-27T07:13:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 client.16563 shutdown 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 msgr2=0x55f788f444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde04010580 tx=0x7fde0400b520 comp rx=0 tx=0).stop 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 shutdown_connections 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:6836/2624187710,v1:172.21.15.69:6837/2624187710] conn(0x55f788f53670 0x55f788fc7640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f788f40e70 0x55f788f44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f788dc1e40 0x55f788f444f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 --2- 172.21.15.16:0/4263172992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f788ead9f0 0x55f788f44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 >> 172.21.15.16:0/4263172992 conn(0x55f788e5b390 msgr2=0x55f788f41ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fddff400700 0 client.16563 ms_handle_reset on v2:172.21.15.69:6836/2624187710 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 shutdown_connections 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.531+0000 7fde2f1bdb80 1 -- 172.21.15.16:0/4263172992 wait complete. 2024-05-27T07:13:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde2f1bdb80 20 client.16563 trim_cache size 0 max 16384 2024-05-27T07:13:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:13:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 msgr2=0x7fde000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 0x7fde000b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde10004fd0 tx=0x7fde10009390 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 msgr2=0x7fde181a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde10007ee0 tx=0x7fde1000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 shutdown_connections 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fde000b2a70 0x7fde000b4f80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde18109840 0x7fde181a1770 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde181045c0 0x7fde1819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 --2- 172.21.15.16:0/93583139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde18108ef0 0x7fde1819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.532+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 >> 172.21.15.16:0/93583139 conn(0x7fde180ffe90 msgr2=0x7fde1810a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.533+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 shutdown_connections 2024-05-27T07:13:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:32.533+0000 7fde20600700 1 -- 172.21.15.16:0/93583139 wait complete. 2024-05-27T07:13:33.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:13:34.037 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:34.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:33 smithi016 ceph-mon[31415]: pgmap v397: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 204 B/s wr, 94 op/s 2024-05-27T07:13:34.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:33 smithi069 ceph-mon[35276]: pgmap v397: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 204 B/s wr, 94 op/s 2024-05-27T07:13:34.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:33 smithi164 ceph-mon[35327]: pgmap v397: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 204 B/s wr, 94 op/s 2024-05-27T07:13:36.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:35 smithi164 ceph-mon[35327]: pgmap v398: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 102 B/s wr, 75 op/s 2024-05-27T07:13:36.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:35 smithi016 ceph-mon[31415]: pgmap v398: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 102 B/s wr, 75 op/s 2024-05-27T07:13:36.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:35 smithi069 ceph-mon[35276]: pgmap v398: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 102 B/s wr, 75 op/s 2024-05-27T07:13:36.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:36 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:13:36.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:36 smithi016 systemd[1]: Stopped Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:36.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:36 smithi016 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service: Consumed 2.472s CPU time. 2024-05-27T07:13:36.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:36 smithi016 systemd[1]: Starting Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:13:37.772 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:37 smithi016 podman[112035]: 2024-05-27 07:13:37.452724096 +0000 UTC m=+0.034896479 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:13:38.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:37 smithi016 ceph-mon[31415]: pgmap v399: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 22 op/s 2024-05-27T07:13:38.055 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:37 smithi016 podman[112035]: 2024-05-27 07:13:37.853938997 +0000 UTC m=+0.436111427 container create 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-05-27T07:13:38.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:37 smithi164 ceph-mon[35327]: pgmap v399: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 22 op/s 2024-05-27T07:13:38.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:37 smithi069 ceph-mon[35276]: pgmap v399: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 22 op/s 2024-05-27T07:13:38.384 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 podman[112035]: 2024-05-27 07:13:38.243363527 +0000 UTC m=+0.825535927 container init 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:13:38.384 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 podman[112035]: 2024-05-27 07:13:38.253416187 +0000 UTC m=+0.835588578 container start 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.393+0000 7fdb99c00700 1 -- 172.21.15.16:0/4040275394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410af40 msgr2=0x7fdb9410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.393+0000 7fdb99c00700 1 --2- 172.21.15.16:0/4040275394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410af40 0x7fdb9410b350 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb800099c0 tx=0x7fdb80009cd0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 -- 172.21.15.16:0/4040275394 shutdown_connections 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 --2- 172.21.15.16:0/4040275394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb9410b890 0x7fdb941131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 --2- 172.21.15.16:0/4040275394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb94106610 0x7fdb94106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 --2- 172.21.15.16:0/4040275394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410af40 0x7fdb9410b350 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 -- 172.21.15.16:0/4040275394 >> 172.21.15.16:0/4040275394 conn(0x7fdb94102020 msgr2=0x7fdb94104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 -- 172.21.15.16:0/4040275394 shutdown_connections 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.394+0000 7fdb99c00700 1 -- 172.21.15.16:0/4040275394 wait complete. 2024-05-27T07:13:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.395+0000 7fdb99c00700 1 Processor -- start 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.395+0000 7fdb99c00700 1 -- start start 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb94106610 0x7fdb9419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb9410af40 0x7fdb9419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94116350 con 0x7fdb9410b890 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941161d0 con 0x7fdb94106610 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941164d0 con 0x7fdb9410af40 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb94106610 0x7fdb9419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41236/0 (socket says 172.21.15.16:41236) 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.396+0000 7fdb99200700 1 -- 172.21.15.16:0/3806996642 learned_addr learned my addr 172.21.15.16:0/3806996642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb93400700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb9410af40 0x7fdb9419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 -- 172.21.15.16:0/3806996642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb9410af40 msgr2=0x7fdb9419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb9410af40 0x7fdb9419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 -- 172.21.15.16:0/3806996642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb94106610 msgr2=0x7fdb9419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb94106610 0x7fdb9419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 -- 172.21.15.16:0/3806996642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941a1c30 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.397+0000 7fdb99200700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007ee0 tx=0x7fdb8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.398+0000 7fdb90c00700 1 -- 172.21.15.16:0/3806996642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004bb0 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.398+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941a1f20 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.398+0000 7fdb90c00700 1 -- 172.21.15.16:0/3806996642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb8c004d50 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.398+0000 7fdb90c00700 1 -- 172.21.15.16:0/3806996642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0058b0 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.398+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941a9840 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.402+0000 7fdb90c00700 1 -- 172.21.15.16:0/3806996642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdb8c01a020 con 0x7fdb9410b890 2024-05-27T07:13:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.403+0000 7fdba9c68b80 1 Processor -- start 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.403+0000 7fdba9c68b80 1 -- start start 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba73ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba74b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555ba66ce40 con 0x5555ba73e410 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5555ba65c9e0 con 0x5555ba5bfe40 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5555ba660a20 con 0x5555ba73e820 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdb90c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 0x7fdb780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.404+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41250/0 (socket says 172.21.15.16:41250) 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 -- 172.21.15.16:0/1674143949 learned_addr learned my addr 172.21.15.16:0/1674143949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93400700 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba73ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93400700 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba74b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb90c00700 1 -- 172.21.15.16:0/3806996642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fdb8c0e20d0 con 0x7fdb9410b890 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb99200700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 0x7fdb780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 -- 172.21.15.16:0/1674143949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 msgr2=0x5555ba74b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba74b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 -- 172.21.15.16:0/1674143949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 msgr2=0x5555ba73ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba73ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.405+0000 7fdb93e00700 1 -- 172.21.15.16:0/1674143949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ba625420 con 0x5555ba73e410 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.406+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bd70 tx=0x7fdb840106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5455294df59aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.406+0000 7fdb89600700 1 -- 172.21.15.16:0/1674143949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84010e10 con 0x5555ba73e410 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.406+0000 7fdb99200700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 0x7fdb780b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0a4fe0 tx=0x7fdb8c0052f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.406+0000 7fdb89600700 1 -- 172.21.15.16:0/1674143949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb8401c910 con 0x5555ba73e410 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.406+0000 7fdb89600700 1 -- 172.21.15.16:0/1674143949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011770 con 0x5555ba73e410 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.407+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1674143949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 msgr2=0x5555ba748ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.407+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bd70 tx=0x7fdb840106b0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1674143949 shutdown_connections 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba74b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba73ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1674143949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ad0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1674143949 >> 172.21.15.16:0/1674143949 conn(0x5555ba61ffc0 msgr2=0x5555ba739d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1674143949 shutdown_connections 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1674143949 wait complete. 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.408+0000 7fdba9c68b80 1 -- start start 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 Processor -- start 2024-05-27T07:13:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba748580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba749000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555ba562fa0 con 0x5555ba73e410 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5555ba5fd760 con 0x5555ba5bfe40 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5555ba74b990 con 0x5555ba73e820 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.409+0000 7fdba9c68b80 10 client.0 _ll_register_callbacks cb 0x5555ba606c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41266/0 (socket says 172.21.15.16:41266) 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 -- 172.21.15.16:0/1397862486 learned_addr learned my addr 172.21.15.16:0/1397862486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93400700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba748580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93400700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba749000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 -- 172.21.15.16:0/1397862486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 msgr2=0x5555ba749000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba749000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 -- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 msgr2=0x5555ba748580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba748580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 -- 172.21.15.16:0/1397862486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ba5ac8d0 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93400700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba749000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bd40 tx=0x7fdb84010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c655a2c927383fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.410+0000 7fdb7e000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401ccc0 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fdb84028470 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8401c910 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb84028670 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdba9c68b80 10 client.16575 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555ba62d060 con 0x5555ba73e410 2024-05-27T07:13:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.411+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb8401c910 con 0x5555ba73e410 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdb840117b0 con 0x5555ba73e410 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 10 client.16575 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 20 client.16575 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba758810 0x5555ba75ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 4 client.16575 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ba66ed90 con 0x5555ba758810 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.412+0000 7fdb93400700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba758810 0x5555ba75ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.414+0000 7fdb93400700 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba758810 0x5555ba75ac30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2975c2ef3ef24e8 server_cookie=a668c776b85e03a3 in_seq=0 out_seq=0 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.414+0000 7fdb7e000700 10 client.16575 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdb7e000700 1 -- 172.21.15.16:0/1397862486 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5555ba66ed90 con 0x5555ba758810 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdb7e000700 10 client.16575 handle_command_reply: tid=0 2024-05-27T07:13:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 client.16575 shutdown 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 msgr2=0x5555ba748ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bd40 tx=0x7fdb84010f70 comp rx=0 tx=0).stop 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 shutdown_connections 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba758810 0x5555ba75ac30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba73e820 0x5555ba749000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba5bfe40 0x5555ba748580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1397862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba748ac0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 >> 172.21.15.16:0/1397862486 conn(0x5555ba61ffc0 msgr2=0x5555ba738ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdb7e000700 0 client.16575 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 shutdown_connections 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.416+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1397862486 wait complete. 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.417+0000 7fdba9c68b80 20 client.16575 trim_cache size 0 max 16384 2024-05-27T07:13:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 Processor -- start 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 -- start start 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba7478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba743d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b2620 con 0x5555ba73e410 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b4160 con 0x5555ba61ffc0 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.434+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b2cf0 con 0x5555ba5bfe40 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41272/0 (socket says 172.21.15.16:41272) 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 -- 172.21.15.16:0/1255737191 learned_addr learned my addr 172.21.15.16:0/1255737191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba7478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93e00700 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba743d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 -- 172.21.15.16:0/1255737191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 msgr2=0x5555ba743d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba743d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 -- 172.21.15.16:0/1255737191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 msgr2=0x5555ba7478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba7478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.435+0000 7fdb93400700 1 -- 172.21.15.16:0/1255737191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ba746af0 con 0x5555ba73e410 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.436+0000 7fdb93400700 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009990 tx=0x7fdb80009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec800bc9e3286c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.436+0000 7fdb88c00700 1 -- 172.21.15.16:0/1255737191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8002d040 con 0x5555ba73e410 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.436+0000 7fdb88c00700 1 -- 172.21.15.16:0/1255737191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb80003680 con 0x5555ba73e410 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.436+0000 7fdb88c00700 1 -- 172.21.15.16:0/1255737191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000f6c0 con 0x5555ba73e410 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1255737191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 msgr2=0x5555ba754c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009990 tx=0x7fdb80009cd0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1255737191 shutdown_connections 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba754c30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba7478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/1255737191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba743d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1255737191 >> 172.21.15.16:0/1255737191 conn(0x5555ba732f60 msgr2=0x5555ba748540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1255737191 shutdown_connections 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.437+0000 7fdba9c68b80 1 -- 172.21.15.16:0/1255737191 wait complete. 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.438+0000 7fdba9c68b80 1 -- start start 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.438+0000 7fdba9c68b80 1 Processor -- start 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba75d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba75ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b3a90 con 0x5555ba73e410 2024-05-27T07:13:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b33c0 con 0x5555ba61ffc0 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b05e0 con 0x5555ba5bfe40 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.439+0000 7fdba9c68b80 10 client.0 _ll_register_callbacks cb 0x5555ba606c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41284/0 (socket says 172.21.15.16:41284) 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 -- 172.21.15.16:0/471085906 learned_addr learned my addr 172.21.15.16:0/471085906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba75d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93e00700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba75ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 -- 172.21.15.16:0/471085906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 msgr2=0x5555ba75d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba75d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 -- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 msgr2=0x5555ba75ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba75ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.440+0000 7fdb93400700 1 -- 172.21.15.16:0/471085906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ba75b4b0 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdb93400700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdb80001eb0 tx=0x7fdb80005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51881d2a3846465e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdb7e000700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb800053b0 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb80003680 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9525+0+0 (secure 0 0 0) 0x7fdb80005550 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb80049050 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdba9c68b80 10 client.16581 fetch_fsmap learned FSMap version 33 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.441+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555ba7c29e0 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb80003680 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdb8000f910 con 0x5555ba73e410 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 20 client.16581 populate_metadata read hostname 'smithi016' 2024-05-27T07:13:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba73e820 0x5555ba7c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 4 client.16581 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.442+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 --> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555ba668570 con 0x5555ba73e820 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.443+0000 7fdb93e00700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba73e820 0x5555ba7c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.444+0000 7fdb93e00700 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba73e820 0x5555ba7c4f40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5c3cee1bda1359 server_cookie=bb2644f1ae5a911d in_seq=0 out_seq=0 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.444+0000 7fdb7e000700 10 client.16581 ms_handle_connect on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.444+0000 7fdb7e000700 1 -- 172.21.15.16:0/471085906 <== mds.0 v2:172.21.15.69:6838/962503578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555ba668570 con 0x5555ba73e820 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.444+0000 7fdb7e000700 10 client.16581 handle_command_reply: tid=0 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.444+0000 7fdba9c68b80 1 client.16581 shutdown 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 msgr2=0x5555ba75ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdb80001eb0 tx=0x7fdb80005040 comp rx=0 tx=0).stop 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 shutdown_connections 2024-05-27T07:13:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:6838/962503578,v1:172.21.15.69:6839/962503578] conn(0x5555ba73e820 0x5555ba7c4f40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ba73e410 0x5555ba75ad80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555ba5bfe40 0x5555ba75d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 --2- 172.21.15.16:0/471085906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555ba61ffc0 0x5555ba75ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 >> 172.21.15.16:0/471085906 conn(0x5555ba732f60 msgr2=0x5555ba747600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdb7e000700 0 client.16581 ms_handle_reset on v2:172.21.15.69:6838/962503578 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 shutdown_connections 2024-05-27T07:13:38.643 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 1 -- 172.21.15.16:0/471085906 wait complete. 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.445+0000 7fdba9c68b80 20 client.16581 trim_cache size 0 max 16384 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 msgr2=0x7fdb780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 0x7fdb780b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0a4fe0 tx=0x7fdb8c0052f0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 msgr2=0x7fdb941a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007ee0 tx=0x7fdb8c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 shutdown_connections 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fdb780b29f0 0x7fdb780b4f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9410b890 0x7fdb941a1590 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb94106610 0x7fdb9419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 --2- 172.21.15.16:0/3806996642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb9410af40 0x7fdb9419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 >> 172.21.15.16:0/3806996642 conn(0x7fdb94102020 msgr2=0x7fdb9410a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 shutdown_connections 2024-05-27T07:13:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:38.446+0000 7fdb99c00700 1 -- 172.21.15.16:0/3806996642 wait complete. 2024-05-27T07:13:38.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 bash[112035]: 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 2024-05-27T07:13:38.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 systemd[1]: Started Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:38.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mds[112067]: -- 172.21.15.16:0/3341128396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d08b1144e0 con 0x55d08b15f180 2024-05-27T07:13:38.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mds[112067]: -- 172.21.15.16:0/3341128396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d08b117a40 con 0x55d08b15f180 2024-05-27T07:13:38.859 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:13:38 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[112061]: starting mds.d at 2024-05-27T07:13:39.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-mon[35327]: daemon mds.d restarted 2024-05-27T07:13:39.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-mon[35327]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:13:39.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.16:6836/1858254783,v1:172.21.15.16:6837/1858254783] up:boot 2024-05-27T07:13:39.062 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7faaff800700 time 2024-05-27T07:13:38.985413+0000 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fab15408ba6] 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.063 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: clone() 2024-05-27T07:13:39.064 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: *** Caught signal (Aborted) ** 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: in thread 7faaff800700 thread_name:md_log_replay 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:38.986+0000 7faaff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7faaff800700 time 2024-05-27T07:13:38.985413+0000 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.065 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fab15408ba6] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.066 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: clone() 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fab13f85d20] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: gsignal() 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: abort() 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fab15408ced] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.067 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 17: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 19: clone() 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:38.989+0000 7faaff800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: in thread 7faaff800700 thread_name:md_log_replay 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.068 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fab13f85d20] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: gsignal() 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: abort() 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fab15408ced] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.069 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 17: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.070 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 19: clone() 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: -1> 2024-05-27T07:13:38.986+0000 7faaff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7faaff800700 time 2024-05-27T07:13:38.985413+0000 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fab15408ba6] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.071 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: clone() 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 0> 2024-05-27T07:13:38.989+0000 7faaff800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: in thread 7faaff800700 thread_name:md_log_replay 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.072 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fab13f85d20] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: gsignal() 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: abort() 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fab15408ced] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.073 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 17: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 19: clone() 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: -1> 2024-05-27T07:13:38.986+0000 7faaff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7faaff800700 time 2024-05-27T07:13:38.985413+0000 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fab15408ba6] 2024-05-27T07:13:39.074 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.075 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: clone() 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 0> 2024-05-27T07:13:38.989+0000 7faaff800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: in thread 7faaff800700 thread_name:md_log_replay 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fab13f85d20] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2: gsignal() 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 3: abort() 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fab15408ced] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fab15408dd1] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fa2206223d] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fa2204d971] 2024-05-27T07:13:39.076 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fa2204bfa0] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fa2204c651] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fa21fe1347] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fa21cdd87e] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fa21ce2e94] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fa21ced969] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fa2208648c] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fa2208f9cb] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 16: (Thread::entry_wrapper()+0x43) [0x7fab153d772d] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 17: (Thread::_entry_func(void*)+0xd) [0x7fab153d7749] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fab13f7b1ca] 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 19: clone() 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:39.077 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71456]: 2024-05-27T07:13:39.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mon[31415]: daemon mds.d restarted 2024-05-27T07:13:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mon[31415]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:13:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.16:6836/1858254783,v1:172.21.15.16:6837/1858254783] up:boot 2024-05-27T07:13:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:39.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:38 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:13:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:38 smithi069 ceph-mon[35276]: daemon mds.d restarted 2024-05-27T07:13:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:38 smithi069 ceph-mon[35276]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:13:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:38 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.16:6836/1858254783,v1:172.21.15.16:6837/1858254783] up:boot 2024-05-27T07:13:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:38 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:39.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:38 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:13:39.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 podman[71506]: 2024-05-27 07:13:39.196408946 +0000 UTC m=+0.052817833 container died 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2) 2024-05-27T07:13:40.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:39 smithi016 ceph-mon[31415]: pgmap v400: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:40.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:39 smithi164 ceph-mon[35327]: pgmap v400: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:40.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 podman[71506]: 2024-05-27 07:13:39.955681302 +0000 UTC m=+0.812090240 container remove 1c763af9dede310bf3c52faeb3441decf51d06e5f0232469e9983869fd6d7e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T07:13:40.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:39 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:13:40.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:39 smithi069 ceph-mon[35276]: pgmap v400: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:40.745 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-27T07:13:40.746 INFO:tasks.fwd_scrub:done joining 2024-05-27T07:13:40.746 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-27T07:13:40.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T07:13:40.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:40 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:13:40.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:40 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.777s CPU time. 2024-05-27T07:13:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 -- 172.21.15.16:0/1338547753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 msgr2=0x7f485c1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 --2- 172.21.15.16:0/1338547753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c1035d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f484c009900 tx=0x7f484c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 -- 172.21.15.16:0/1338547753 shutdown_connections 2024-05-27T07:13:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 --2- 172.21.15.16:0/1338547753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c104180 0x7f485c1104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 --2- 172.21.15.16:0/1338547753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c109ec0 0x7f485c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 --2- 172.21.15.16:0/1338547753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c1035d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 -- 172.21.15.16:0/1338547753 >> 172.21.15.16:0/1338547753 conn(0x7f485c07a210 msgr2=0x7f485c07a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 -- 172.21.15.16:0/1338547753 shutdown_connections 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.116+0000 7f4861c00640 1 -- 172.21.15.16:0/1338547753 wait complete. 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.117+0000 7f4861c00640 1 Processor -- start 2024-05-27T07:13:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.117+0000 7f4861c00640 1 -- start start 2024-05-27T07:13:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c104180 0x7f485c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 0x7f485c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f485c113b20 con 0x7f485c109a20 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1139a0 con 0x7f485c104180 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f4861c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f485c113ca0 con 0x7f485c109ec0 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f485b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f485b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41320/0 (socket says 172.21.15.16:41320) 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.118+0000 7f485b400640 1 -- 172.21.15.16:0/3485257588 learned_addr learned my addr 172.21.15.16:0/3485257588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485be00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c104180 0x7f485c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f4861200640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 0x7f485c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 -- 172.21.15.16:0/3485257588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 msgr2=0x7f485c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 0x7f485c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 -- 172.21.15.16:0/3485257588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c104180 msgr2=0x7f485c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c104180 0x7f485c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 -- 172.21.15.16:0/3485257588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c19f3a0 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f4861200640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 0x7f485c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f485b400640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f484800d740 tx=0x7f484800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848017070 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.119+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f484800de50 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.120+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c19f690 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.120+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848003bb0 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.120+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a7010 con 0x7f485c109a20 2024-05-27T07:13:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.121+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c069d00 con 0x7f485c109a20 2024-05-27T07:13:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.124+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4848003d50 con 0x7f485c109a20 2024-05-27T07:13:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.126+0000 7f4858c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 0x7f48400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.126+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7f48480e24a0 con 0x7f485c109a20 2024-05-27T07:13:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.126+0000 7f485be00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 0x7f48400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.127+0000 7f485be00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 0x7f48400b4e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f484c02a910 tx=0x7f484c000bc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.131+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48480a9a90 con 0x7f485c109a20 2024-05-27T07:13:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.343+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f485c107f30 con 0x7f485c109a20 2024-05-27T07:13:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.347+0000 7f4858c00640 1 -- 172.21.15.16:0/3485257588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+25135 (secure 0 0 0) 0x7f48480a9390 con 0x7f485c109a20 2024-05-27T07:13:41.339 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:13:41.339 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":107,"fsid":"3c4673a4-1bf4-11ef-bc9b-c7b262605968","created":"2024-05-27T06:43:14.821531+0000","modified":"2024-05-27T07:13:38.597377+0000","last_up_change":"2024-05-27T06:53:40.100008+0000","last_in_change":"2024-05-27T06:53:14.065652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T06:47:58.074809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T06:55:03.523883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T06:55:08.339180+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T06:55:24.196525+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee8028c3-dc9e-4ece-a30c-2129ffe7ef4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6803","nonce":4171777191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6805","nonce":4171777191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6809","nonce":4171777191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4171777191},{"type":"v1","addr":"172.21.15.16:6807","nonce":4171777191}]},"public_addr":"172.21.15.16:6803/4171777191","cluster_addr":"172.21.15.16:6805/4171777191","heartbeat_back_addr":"172.21.15.16:6809/4171777191","heartbeat_front_addr":"172.21.15.16:6807/4171777191","state":["exists","up"]},{"osd":1,"uuid":"6f0fa1e6-b4a7-48a6-b89d-c5612a23e676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6801","nonce":1028348959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6803","nonce":1028348959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6807","nonce":1028348959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1028348959},{"type":"v1","addr":"172.21.15.69:6805","nonce":1028348959}]},"public_addr":"172.21.15.69:6801/1028348959","cluster_addr":"172.21.15.69:6803/1028348959","heartbeat_back_addr":"172.21.15.69:6807/1028348959","heartbeat_front_addr":"172.21.15.69:6805/1028348959","state":["exists","up"]},{"osd":2,"uuid":"ebccb12e-a7d2-4907-9140-78f8ab1994bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6801","nonce":3071758957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6803","nonce":3071758957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6807","nonce":3071758957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3071758957},{"type":"v1","addr":"172.21.15.164:6805","nonce":3071758957}]},"public_addr":"172.21.15.164:6801/3071758957","cluster_addr":"172.21.15.164:6803/3071758957","heartbeat_back_addr":"172.21.15.164:6807/3071758957","heartbeat_front_addr":"172.21.15.164:6805/3071758957","state":["exists","up"]},{"osd":3,"uuid":"0b1c5db9-adc3-444b-9eca-3599f6e32f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6811","nonce":3551792310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6813","nonce":3551792310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6817","nonce":3551792310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3551792310},{"type":"v1","addr":"172.21.15.16:6815","nonce":3551792310}]},"public_addr":"172.21.15.16:6811/3551792310","cluster_addr":"172.21.15.16:6813/3551792310","heartbeat_back_addr":"172.21.15.16:6817/3551792310","heartbeat_front_addr":"172.21.15.16:6815/3551792310","state":["exists","up"]},{"osd":4,"uuid":"c25a93d3-53d0-4155-8981-50f926759f79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6809","nonce":823147772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6811","nonce":823147772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6815","nonce":823147772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":823147772},{"type":"v1","addr":"172.21.15.69:6813","nonce":823147772}]},"public_addr":"172.21.15.69:6809/823147772","cluster_addr":"172.21.15.69:6811/823147772","heartbeat_back_addr":"172.21.15.69:6815/823147772","heartbeat_front_addr":"172.21.15.69:6813/823147772","state":["exists","up"]},{"osd":5,"uuid":"e85ea8a0-f98e-416d-9d63-ed816d3afb04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6809","nonce":3562698825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6811","nonce":3562698825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6815","nonce":3562698825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3562698825},{"type":"v1","addr":"172.21.15.164:6813","nonce":3562698825}]},"public_addr":"172.21.15.164:6809/3562698825","cluster_addr":"172.21.15.164:6811/3562698825","heartbeat_back_addr":"172.21.15.164:6815/3562698825","heartbeat_front_addr":"172.21.15.164:6813/3562698825","state":["exists","up"]},{"osd":6,"uuid":"be746ea8-a314-4561-9bf5-bc7d7944ffce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6819","nonce":1434782901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6821","nonce":1434782901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6825","nonce":1434782901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1434782901},{"type":"v1","addr":"172.21.15.16:6823","nonce":1434782901}]},"public_addr":"172.21.15.16:6819/1434782901","cluster_addr":"172.21.15.16:6821/1434782901","heartbeat_back_addr":"172.21.15.16:6825/1434782901","heartbeat_front_addr":"172.21.15.16:6823/1434782901","state":["exists","up"]},{"osd":7,"uuid":"a4dced21-90cd-4a3d-8823-02824ca1c42a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6817","nonce":125788904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6819","nonce":125788904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6823","nonce":125788904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":125788904},{"type":"v1","addr":"172.21.15.69:6821","nonce":125788904}]},"public_addr":"172.21.15.69:6817/125788904","cluster_addr":"172.21.15.69:6819/125788904","heartbeat_back_addr":"172.21.15.69:6823/125788904","heartbeat_front_addr":"172.21.15.69:6821/125788904","state":["exists","up"]},{"osd":8,"uuid":"d3dc3371-6d3b-403e-a4d3-4bfb2dbaacfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6817","nonce":4237870285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6819","nonce":4237870285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6823","nonce":4237870285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":4237870285},{"type":"v1","addr":"172.21.15.164:6821","nonce":4237870285}]},"public_addr":"172.21.15.164:6817/4237870285","cluster_addr":"172.21.15.164:6819/4237870285","heartbeat_back_addr":"172.21.15.164:6823/4237870285","heartbeat_front_addr":"172.21.15.164:6821/4237870285","state":["exists","up"]},{"osd":9,"uuid":"6e4f0c06-7a28-44a5-8ee0-84370c455177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6827","nonce":1571284516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6829","nonce":1571284516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6833","nonce":1571284516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1571284516},{"type":"v1","addr":"172.21.15.16:6831","nonce":1571284516}]},"public_addr":"172.21.15.16:6827/1571284516","cluster_addr":"172.21.15.16:6829/1571284516","heartbeat_back_addr":"172.21.15.16:6833/1571284516","heartbeat_front_addr":"172.21.15.16:6831/1571284516","state":["exists","up"]},{"osd":10,"uuid":"c705b502-5d5e-4b43-89db-40a14acec587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6825","nonce":1283060257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6827","nonce":1283060257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6831","nonce":1283060257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1283060257},{"type":"v1","addr":"172.21.15.69:6829","nonce":1283060257}]},"public_addr":"172.21.15.69:6825/1283060257","cluster_addr":"172.21.15.69:6827/1283060257","heartbeat_back_addr":"172.21.15.69:6831/1283060257","heartbeat_front_addr":"172.21.15.69:6829/1283060257","state":["exists","up"]},{"osd":11,"uuid":"e1b29f9e-85de-403e-bc32-f10ad4d089cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6825","nonce":3936768359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6827","nonce":3936768359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6831","nonce":3936768359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3936768359},{"type":"v1","addr":"172.21.15.164:6829","nonce":3936768359}]},"public_addr":"172.21.15.164:6825/3936768359","cluster_addr":"172.21.15.164:6827/3936768359","heartbeat_back_addr":"172.21.15.164:6831/3936768359","heartbeat_front_addr":"172.21.15.164:6829/3936768359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:46:46.667476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:21.948248+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:47:56.594108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:48:36.343525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:14.019757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:49:50.002288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:50:29.562130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:06.995898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:51:43.727592+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:52:23.212376+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:00.980979+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T06:53:38.160342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/702672487":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6800/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3562581452":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/1801942445":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/2589421123":"2024-05-28T06:44:37.876629+0000","172.21.15.16:6801/2390178341":"2024-05-28T06:44:37.876629+0000","172.21.15.16:0/454643589":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3708593763":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3327639895":"2024-05-28T06:44:00.893187+0000","172.21.15.16:0/3317326035":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/3906409731":"2024-05-28T07:00:53.625719+0000","172.21.15.16:0/4243270602":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6801/2755801148":"2024-05-28T07:00:53.625719+0000","172.21.15.16:6801/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/4120962675":"2024-05-28T06:44:00.893187+0000","172.21.15.16:6800/2390178341":"2024-05-28T06:44:37.876629+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 msgr2=0x7f48400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 0x7f48400b4e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f484c02a910 tx=0x7f484c000bc0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 msgr2=0x7f485c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f484800d740 tx=0x7f484800dc00 comp rx=0 tx=0).stop 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 shutdown_connections 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f48400b2950 0x7f48400b4e60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f485c109ec0 0x7f485c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f485c109a20 0x7f485c19aaa0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 --2- 172.21.15.16:0/3485257588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c104180 0x7f485c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 >> 172.21.15.16:0/3485257588 conn(0x7f485c07a210 msgr2=0x7f485c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.350+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 shutdown_connections 2024-05-27T07:13:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.351+0000 7f4861c00640 1 -- 172.21.15.16:0/3485257588 wait complete. 2024-05-27T07:13:41.370 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T06:47:58.074809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T06:55:03.523883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T06:55:08.339180+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T06:55:24.196525+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T07:13:41.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 -- 172.21.15.16:0/2008830764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e6010a760 msgr2=0x7f5e6010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 --2- 172.21.15.16:0/2008830764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e6010a760 0x7f5e6010ab70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009ca0 tx=0x7f5e5002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 -- 172.21.15.16:0/2008830764 shutdown_connections 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 --2- 172.21.15.16:0/2008830764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 0x7f5e6011b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 --2- 172.21.15.16:0/2008830764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e60114110 0x7f5e60114520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 --2- 172.21.15.16:0/2008830764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e6010a760 0x7f5e6010ab70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 -- 172.21.15.16:0/2008830764 >> 172.21.15.16:0/2008830764 conn(0x7f5e6006ed70 msgr2=0x7f5e6006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 -- 172.21.15.16:0/2008830764 shutdown_connections 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.680+0000 7f5e64e00640 1 -- 172.21.15.16:0/2008830764 wait complete. 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.681+0000 7f5e64e00640 1 Processor -- start 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 -- start start 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e6010a760 0x7f5e601a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 0x7f5e601a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6011e6f0 con 0x7f5e60114110 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6011e570 con 0x7f5e60114b40 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e64e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6011e870 con 0x7f5e6010a760 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e55400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e55400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41342/0 (socket says 172.21.15.16:41342) 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.682+0000 7f5e55400640 1 -- 172.21.15.16:0/1813862555 learned_addr learned my addr 172.21.15.16:0/1813862555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e5de00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e6010a760 0x7f5e601a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e5e800640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 0x7f5e601a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 -- 172.21.15.16:0/1813862555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e6010a760 msgr2=0x7f5e601a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e6010a760 0x7f5e601a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 -- 172.21.15.16:0/1813862555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 msgr2=0x7f5e601a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 0x7f5e601a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 -- 172.21.15.16:0/1813862555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601a9f30 con 0x7f5e60114110 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e55400640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d770 tx=0x7f5e4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.683+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48017070 con 0x7f5e60114110 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.684+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e48009c30 con 0x7f5e60114110 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.684+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800f620 con 0x7f5e60114110 2024-05-27T07:13:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.684+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601aa220 con 0x7f5e60114110 2024-05-27T07:13:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.684+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601aa720 con 0x7f5e60114110 2024-05-27T07:13:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.685+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e60069e30 con 0x7f5e60114110 2024-05-27T07:13:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.687+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5e4801a070 con 0x7f5e60114110 2024-05-27T07:13:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.692+0000 7f5e57400640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 0x7f5e440b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.692+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7f5e480e2530 con 0x7f5e60114110 2024-05-27T07:13:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.692+0000 7f5e5de00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 0x7f5e440b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.693+0000 7f5e5de00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 0x7f5e440b4ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000f80 tx=0x7f5e500093f0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:41.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.694+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e601aa720 con 0x7f5e60114110 2024-05-27T07:13:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.909+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e60116700 con 0x7f5e60114110 2024-05-27T07:13:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.910+0000 7f5e57400640 1 -- 172.21.15.16:0/1813862555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f5e48009de0 con 0x7f5e60114110 2024-05-27T07:13:41.902 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-27T07:13:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 msgr2=0x7f5e440b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 0x7f5e440b4ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000f80 tx=0x7f5e500093f0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 msgr2=0x7f5e601a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d770 tx=0x7f5e4800dc30 comp rx=0 tx=0).stop 2024-05-27T07:13:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 shutdown_connections 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f5e440b29e0 0x7f5e440b4ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e60114b40 0x7f5e601a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60114110 0x7f5e601a5630 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 --2- 172.21.15.16:0/1813862555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e6010a760 0x7f5e601a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.914+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 >> 172.21.15.16:0/1813862555 conn(0x7f5e6006ed70 msgr2=0x7f5e6010bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.915+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 shutdown_connections 2024-05-27T07:13:41.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:41.915+0000 7f5e64e00640 1 -- 172.21.15.16:0/1813862555 wait complete. 2024-05-27T07:13:41.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T07:13:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 -- 172.21.15.16:0/3252006887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c110250 msgr2=0x7fd82c1126b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 --2- 172.21.15.16:0/3252006887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c110250 0x7fd82c1126b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd82402a950 comp rx=0 tx=0).stop 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 -- 172.21.15.16:0/3252006887 shutdown_connections 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 --2- 172.21.15.16:0/3252006887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c110250 0x7fd82c1126b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 --2- 172.21.15.16:0/3252006887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c10d8f0 0x7fd82c10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 --2- 172.21.15.16:0/3252006887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c075660 0x7fd82c075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 -- 172.21.15.16:0/3252006887 >> 172.21.15.16:0/3252006887 conn(0x7fd82c0ff0e0 msgr2=0x7fd82c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.243+0000 7fd832200640 1 -- 172.21.15.16:0/3252006887 shutdown_connections 2024-05-27T07:13:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.244+0000 7fd832200640 1 -- 172.21.15.16:0/3252006887 wait complete. 2024-05-27T07:13:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 Processor -- start 2024-05-27T07:13:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 -- start start 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c075660 0x7fd82c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c110250 0x7fd82c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c115d20 con 0x7fd82c10d8f0 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c115ba0 con 0x7fd82c110250 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.245+0000 7fd832200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c115ea0 con 0x7fd82c075660 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51526/0 (socket says 172.21.15.16:51526) 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 -- 172.21.15.16:0/2883780120 learned_addr learned my addr 172.21.15.16:0/2883780120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd830e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c075660 0x7fd82c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd831800640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c110250 0x7fd82c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 -- 172.21.15.16:0/2883780120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c075660 msgr2=0x7fd82c19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c075660 0x7fd82c19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 -- 172.21.15.16:0/2883780120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c110250 msgr2=0x7fd82c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c110250 0x7fd82c1a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.246+0000 7fd823e00640 1 -- 172.21.15.16:0/2883780120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c1a1550 con 0x7fd82c10d8f0 2024-05-27T07:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd823e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c017030 con 0x7fd82c10d8f0 2024-05-27T07:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd81c004020 con 0x7fd82c10d8f0 2024-05-27T07:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c1a1840 con 0x7fd82c10d8f0 2024-05-27T07:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0055d0 con 0x7fd82c10d8f0 2024-05-27T07:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.247+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a91c0 con 0x7fd82c10d8f0 2024-05-27T07:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.248+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c0692d0 con 0x7fd82c10d8f0 2024-05-27T07:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.253+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd81c009a30 con 0x7fd82c10d8f0 2024-05-27T07:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.254+0000 7fd82a000640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 0x7fd80c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.255+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7fd81c0e2a20 con 0x7fd82c10d8f0 2024-05-27T07:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.255+0000 7fd830e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 0x7fd80c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.256+0000 7fd830e00640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 0x7fd80c0b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd814005fd0 tx=0x7fd814005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.257+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd81c0aa090 con 0x7fd82c10d8f0 2024-05-27T07:13:42.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:41 smithi164 ceph-mon[35327]: pgmap v402: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:42.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:41 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3485257588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:13:42.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:41 smithi016 ceph-mon[31415]: pgmap v402: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:42.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:41 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3485257588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:13:42.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:41 smithi069 ceph-mon[35276]: pgmap v402: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:42.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:41 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3485257588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:13:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.479+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd82c1a93f0 con 0x7fd82c10d8f0 2024-05-27T07:13:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.481+0000 7fd82a000640 1 -- 172.21.15.16:0/2883780120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7fd81c0a9990 con 0x7fd82c10d8f0 2024-05-27T07:13:42.472 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.483+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 msgr2=0x7fd80c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.483+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 0x7fd80c0b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd814005fd0 tx=0x7fd814005850 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.483+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 msgr2=0x7fd82c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.483+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00de10 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 shutdown_connections 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7fd80c0b29d0 0x7fd80c0b4ee0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd82c110250 0x7fd82c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd82c10d8f0 0x7fd82c19cca0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 --2- 172.21.15.16:0/2883780120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd82c075660 0x7fd82c19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 >> 172.21.15.16:0/2883780120 conn(0x7fd82c0ff0e0 msgr2=0x7fd82c111170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 shutdown_connections 2024-05-27T07:13:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.484+0000 7fd832200640 1 -- 172.21.15.16:0/2883780120 wait complete. 2024-05-27T07:13:42.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.807+0000 7fa001800640 1 -- 172.21.15.16:0/2615018930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc075400 msgr2=0x7f9ffc075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.807+0000 7fa001800640 1 --2- 172.21.15.16:0/2615018930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc075400 0x7f9ffc075810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009900 tx=0x7f9fec02a480 comp rx=0 tx=0).stop 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 -- 172.21.15.16:0/2615018930 shutdown_connections 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 --2- 172.21.15.16:0/2615018930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc110330 0x7f9ffc112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 --2- 172.21.15.16:0/2615018930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc10fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 --2- 172.21.15.16:0/2615018930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc075400 0x7f9ffc075810 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 -- 172.21.15.16:0/2615018930 >> 172.21.15.16:0/2615018930 conn(0x7f9ffc0ff2c0 msgr2=0x7f9ffc1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 -- 172.21.15.16:0/2615018930 shutdown_connections 2024-05-27T07:13:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.808+0000 7fa001800640 1 -- 172.21.15.16:0/2615018930 wait complete. 2024-05-27T07:13:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.809+0000 7fa001800640 1 Processor -- start 2024-05-27T07:13:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.809+0000 7fa001800640 1 -- start start 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc075400 0x7f9ffc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ffc110330 0x7f9ffc1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc115d40 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc115bc0 con 0x7f9ffc075400 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7fa001800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc115ec0 con 0x7f9ffc110330 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51554/0 (socket says 172.21.15.16:51554) 2024-05-27T07:13:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 -- 172.21.15.16:0/126864505 learned_addr learned my addr 172.21.15.16:0/126864505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffbe00640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc075400 0x7f9ffc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 -- 172.21.15.16:0/126864505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ffc110330 msgr2=0x7f9ffc1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ffc110330 0x7f9ffc1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 -- 172.21.15.16:0/126864505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc075400 msgr2=0x7f9ffc19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc075400 0x7f9ffc19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.810+0000 7f9ffb400640 1 -- 172.21.15.16:0/126864505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc1a1640 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7f9ffb400640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d950 tx=0x7f9fe800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8017070 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fe8009c30 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc1a1930 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800f620 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.811+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc1a92b0 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.813+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc06a150 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.815+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9fe801a070 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.817+0000 7f9ff8c00640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 0x7f9fe00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.817+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7f9fe80e24d0 con 0x7f9ffc10d9d0 2024-05-27T07:13:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.817+0000 7f9ffbe00640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 0x7f9fe00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.818+0000 7f9ffbe00640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 0x7f9fe00b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009230 tx=0x7f9fec005fd0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:42.819+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fe8009de0 con 0x7f9ffc10d9d0 2024-05-27T07:13:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.043+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ffc1a94e0 con 0x7f9ffc10d9d0 2024-05-27T07:13:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.044+0000 7f9ff8c00640 1 -- 172.21.15.16:0/126864505 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f9fe80a9b40 con 0x7f9ffc10d9d0 2024-05-27T07:13:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-27T07:13:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 msgr2=0x7f9fe00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 0x7f9fe00b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009230 tx=0x7f9fec005fd0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 msgr2=0x7f9ffc19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d950 tx=0x7f9fe800de10 comp rx=0 tx=0).stop 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 shutdown_connections 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9fe00b29d0 0x7f9fe00b4ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ffc110330 0x7f9ffc1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc10d9d0 0x7f9ffc19cd40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 --2- 172.21.15.16:0/126864505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ffc075400 0x7f9ffc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 >> 172.21.15.16:0/126864505 conn(0x7f9ffc0ff2c0 msgr2=0x7f9ffc110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.047+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 shutdown_connections 2024-05-27T07:13:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.048+0000 7fa001800640 1 -- 172.21.15.16:0/126864505 wait complete. 2024-05-27T07:13:43.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T07:13:43.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:42 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1813862555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:42 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/2883780120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:42 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1813862555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:42 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/2883780120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 -- 172.21.15.16:0/2770693778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d410d2e0 msgr2=0x7f08d410f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 --2- 172.21.15.16:0/2770693778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d410d2e0 0x7f08d410f700 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009ca0 tx=0x7f08cc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 -- 172.21.15.16:0/2770693778 shutdown_connections 2024-05-27T07:13:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 --2- 172.21.15.16:0/2770693778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d410d2e0 0x7f08d410f700 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 --2- 172.21.15.16:0/2770693778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d4102a50 0x7f08d4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 --2- 172.21.15.16:0/2770693778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4108980 0x7f08d4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 -- 172.21.15.16:0/2770693778 >> 172.21.15.16:0/2770693778 conn(0x7f08d40fe030 msgr2=0x7f08d4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 -- 172.21.15.16:0/2770693778 shutdown_connections 2024-05-27T07:13:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.376+0000 7f08d3200640 1 -- 172.21.15.16:0/2770693778 wait complete. 2024-05-27T07:13:43.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:42 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1813862555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.369 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:42 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/2883780120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.377+0000 7f08d3200640 1 Processor -- start 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.377+0000 7f08d3200640 1 -- start start 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4102a50 0x7f08d419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d410d2e0 0x7f08d419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112a60 con 0x7f08d4108980 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41128e0 con 0x7f08d4102a50 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112be0 con 0x7f08d410d2e0 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51566/0 (socket says 172.21.15.16:51566) 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d1400640 1 -- 172.21.15.16:0/1208277164 learned_addr learned my addr 172.21.15.16:0/1208277164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.378+0000 7f08d1e00640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4102a50 0x7f08d419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d2800640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d410d2e0 0x7f08d419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 -- 172.21.15.16:0/1208277164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d410d2e0 msgr2=0x7f08d419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d410d2e0 0x7f08d419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 -- 172.21.15.16:0/1208277164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4102a50 msgr2=0x7f08d419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4102a50 0x7f08d419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 -- 172.21.15.16:0/1208277164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d419f350 con 0x7f08d4108980 2024-05-27T07:13:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d1400640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005d00 tx=0x7f08c000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0017070 con 0x7f08d4108980 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d419f640 con 0x7f08d4108980 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08c0009c30 con 0x7f08d4108980 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.379+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000f620 con 0x7f08d4108980 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.380+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d419fb40 con 0x7f08d4108980 2024-05-27T07:13:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.381+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4069d00 con 0x7f08d4108980 2024-05-27T07:13:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.385+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f08c001a070 con 0x7f08d4108980 2024-05-27T07:13:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.386+0000 7f08caa00640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 0x7f08b80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.387+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7f08c00e24b0 con 0x7f08d4108980 2024-05-27T07:13:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.387+0000 7f08d1e00640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 0x7f08b80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.388+0000 7f08d1e00640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 0x7f08b80b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005fd0 tx=0x7f08c4005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.388+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08d419fb40 con 0x7f08d4108980 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.603+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f08d41a70c0 con 0x7f08d4108980 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.604+0000 7f08caa00640 1 -- 172.21.15.16:0/1208277164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) ==== 103+0+11 (secure 0 0 0) 0x7f08c0009de0 con 0x7f08d4108980 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 msgr2=0x7f08b80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 0x7f08b80b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005fd0 tx=0x7f08c4005850 comp rx=0 tx=0).stop 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 msgr2=0x7f08d419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005d00 tx=0x7f08c000dde0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 shutdown_connections 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f08b80b2950 0x7f08b80b4e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d410d2e0 0x7f08d419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d4108980 0x7f08d419aa50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 --2- 172.21.15.16:0/1208277164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08d4102a50 0x7f08d419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 >> 172.21.15.16:0/1208277164 conn(0x7f08d40fe030 msgr2=0x7f08d4069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 shutdown_connections 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.607+0000 7f08d3200640 1 -- 172.21.15.16:0/1208277164 wait complete. 2024-05-27T07:13:43.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-27T07:13:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 -- 172.21.15.16:0/2162448040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b4106600 msgr2=0x7f20b4112980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 --2- 172.21.15.16:0/2162448040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b4106600 0x7f20b4112980 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009ca0 tx=0x7f20ac02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 -- 172.21.15.16:0/2162448040 shutdown_connections 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 --2- 172.21.15.16:0/2162448040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b4106600 0x7f20b4112980 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 --2- 172.21.15.16:0/2162448040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b4105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 --2- 172.21.15.16:0/2162448040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b410be60 0x7f20b4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 -- 172.21.15.16:0/2162448040 >> 172.21.15.16:0/2162448040 conn(0x7f20b41016b0 msgr2=0x7f20b4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 -- 172.21.15.16:0/2162448040 shutdown_connections 2024-05-27T07:13:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.925+0000 7f20b9c00640 1 -- 172.21.15.16:0/2162448040 wait complete. 2024-05-27T07:13:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.926+0000 7f20b9c00640 1 Processor -- start 2024-05-27T07:13:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 -- start start 2024-05-27T07:13:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4106600 0x7f20b41a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115f70 con 0x7f20b410be60 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115df0 con 0x7f20b4106600 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41160f0 con 0x7f20b410c300 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51586/0 (socket says 172.21.15.16:51586) 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.927+0000 7f20b2000640 1 -- 172.21.15.16:0/3661418490 learned_addr learned my addr 172.21.15.16:0/3661418490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2a00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4106600 0x7f20b41a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b3400640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 -- 172.21.15.16:0/3661418490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 msgr2=0x7f20b419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 -- 172.21.15.16:0/3661418490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4106600 msgr2=0x7f20b41a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4106600 0x7f20b41a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 -- 172.21.15.16:0/3661418490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b419ce80 con 0x7f20b410be60 2024-05-27T07:13:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b3400640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:13:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.928+0000 7f20b2000640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005d00 tx=0x7f20a000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.929+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b410be60 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.929+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20a0009c30 con 0x7f20b410be60 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.929+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b419d110 con 0x7f20b410be60 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.929+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000f620 con 0x7f20b410be60 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.929+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a9370 con 0x7f20b410be60 2024-05-27T07:13:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.930+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b4107140 con 0x7f20b410be60 2024-05-27T07:13:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.934+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f20a001a070 con 0x7f20b410be60 2024-05-27T07:13:43.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.936+0000 7f20ab400640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 0x7f20980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:13:43.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.936+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9437+0+0 (secure 0 0 0) 0x7f20a00e23c0 con 0x7f20b410be60 2024-05-27T07:13:43.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.936+0000 7f20b2a00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 0x7f20980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:13:43.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.937+0000 7f20b2a00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 0x7f20980b4ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:13:43.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:43.941+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20a0009de0 con 0x7f20b410be60 2024-05-27T07:13:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.208+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f20b410df70 con 0x7f20b410be60 2024-05-27T07:13:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.247+0000 7f20ab400640 1 -- 172.21.15.16:0/3661418490 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+816174 (secure 0 0 0) 0x7f20a00a9a30 con 0x7f20b410be60 2024-05-27T07:13:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-27T07:13:44.243 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:13:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-05-27T07:13:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 15297, 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1617637164 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1617637164 2024-05-27T07:13:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/1617637164", 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-05-27T07:00:53.626452+0000", 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 25036, 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-27T07:13:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-27T07:13:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-27T07:13:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T07:13:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-27T07:13:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-27T07:13:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T07:13:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-27T07:13:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-27T07:13:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-27T07:13:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-27T07:13:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T07:13:44.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T07:13:44.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T07:13:44.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-27T07:13:44.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-27T07:13:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.304 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T07:13:44.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-27T07:13:44.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-27T07:13:44.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T07:13:44.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-27T07:13:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-27T07:13:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.324 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-27T07:13:44.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-27T07:13:44.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T07:13:44.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-27T07:13:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.357 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-27T07:13:44.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-27T07:13:44.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T07:13:44.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T07:13:44.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.372 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T07:13:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T07:13:44.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.413 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.417 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-27T07:13:44.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:44.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-27T07:13:44.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.429 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:44.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.433 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:43 smithi069 ceph-mon[35276]: pgmap v403: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/126864505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:43 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1208277164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:43 smithi164 ceph-mon[35327]: pgmap v403: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/126864505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:43 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1208277164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.443 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-27T07:13:44.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-27T07:13:44.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.447 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.449 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-27T07:13:44.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-27T07:13:44.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-27T07:13:44.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.471 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-27T07:13:44.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.474 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-27T07:13:44.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-27T07:13:44.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.481 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.490 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.502 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.503 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.504 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T07:13:44.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.522 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.523 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.526 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.529 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.530 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.531 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.531 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.546 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.549 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.552 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.556 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-27T07:13:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:44.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.585 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-27T07:13:44.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:44.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.603 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-27T07:13:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-27T07:13:44.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.607 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.608 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.609 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:44.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.629 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-27T07:13:44.636 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-27T07:13:44.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.647 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-27T07:13:44.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-27T07:13:44.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.650 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.674 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.684 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.695 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-27T07:13:44.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.729 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-27T07:13:44.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.735 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T07:13:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 35066, 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.752 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.753 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.756 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-27T07:13:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-27T07:13:44.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.764 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-27T07:13:44.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-27T07:13:44.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T07:13:44.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T07:13:44.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:44.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:44.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.777 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.780 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-27T07:13:44.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.786 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.787 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-27T07:13:44.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-27T07:13:44.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.794 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.795 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T07:13:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T07:13:44.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T07:13:44.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-27T07:13:44.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T07:13:44.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-27T07:13:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-27T07:13:44.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-27T07:13:44.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-27T07:13:44.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T07:13:44.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T07:13:44.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T07:13:44.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.831 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.833 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-27T07:13:44.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T07:13:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-27T07:13:44.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T07:13:44.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.859 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-27T07:13:44.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.866 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T07:13:44.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T07:13:44.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.872 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T07:13:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T07:13:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T07:13:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T07:13:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T07:13:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-27T07:13:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-27T07:13:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.274+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 msgr2=0x7f20980b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.274+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 0x7f20980b4ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4005850 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.274+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 msgr2=0x7f20b41a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.274+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005d00 tx=0x7f20a000dde0 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.274+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 shutdown_connections 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f20980b29e0 0x7f20980b4ef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20b410c300 0x7f20b419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b410be60 0x7f20b41a2b10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 --2- 172.21.15.16:0/3661418490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4106600 0x7f20b41a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 >> 172.21.15.16:0/3661418490 conn(0x7f20b41016b0 msgr2=0x7f20b4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 shutdown_connections 2024-05-27T07:13:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:13:44.275+0000 7f20b9c00640 1 -- 172.21.15.16:0/3661418490 wait complete. 2024-05-27T07:13:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:43 smithi016 ceph-mon[31415]: pgmap v403: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/126864505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:43 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1208277164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-27T07:13:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-27T07:13:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-27T07:13:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-27T07:13:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T07:13:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-27T07:13:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-27T07:13:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-27T07:13:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-27T07:13:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-27T07:13:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-27T07:13:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-27T07:13:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-27T07:13:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-27T07:13:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-27T07:13:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-27T07:13:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-27T07:13:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-27T07:13:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-27T07:13:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-27T07:13:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-27T07:13:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T07:13:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-27T07:13:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-27T07:13:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-27T07:13:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-27T07:13:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-27T07:13:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-27T07:13:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-27T07:13:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-27T07:13:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-27T07:13:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T07:13:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-27T07:13:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-27T07:13:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T07:13:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-05-27T07:13:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-27T07:13:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-27T07:13:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T07:13:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-27T07:13:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-27T07:13:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-27T07:13:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T07:13:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-27T07:13:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T07:13:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-27T07:13:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T07:13:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T07:13:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T07:13:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T07:13:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T07:13:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T07:13:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T07:13:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T07:13:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T07:13:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T07:13:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T07:13:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-27T07:13:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-27T07:13:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-27T07:13:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-27T07:13:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-27T07:13:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-27T07:13:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-27T07:13:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-27T07:13:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-27T07:13:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-27T07:13:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-27T07:13:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-27T07:13:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-27T07:13:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-27T07:13:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T07:13:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-27T07:13:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-27T07:13:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-27T07:13:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-27T07:13:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-27T07:13:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-27T07:13:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-27T07:13:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-27T07:13:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-27T07:13:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-27T07:13:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-27T07:13:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-27T07:13:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.747 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-05-27T07:13:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T07:13:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T07:13:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-27T07:13:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-27T07:13:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-27T07:13:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-27T07:13:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-27T07:13:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-27T07:13:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-05-27T07:13:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-27T07:13:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-27T07:13:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-27T07:13:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 91, 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3994731719 2024-05-27T07:13:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 500120961 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-27T07:13:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1609844656 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3338190617 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3539232335 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1661447081 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-27T07:13:45.784 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:13:45.846 DEBUG:tasks.check_counter:Getting stats from a 2024-05-27T07:13:45.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-27T07:13:45.848 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:44 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/3661418490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T07:13:45.848 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:44 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/3661418490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T07:13:45.848 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:44 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/3661418490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T07:13:46.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:45 smithi164 ceph-mon[35327]: pgmap v404: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:46.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:45 smithi016 ceph-mon[31415]: pgmap v404: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:46.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:45 smithi069 ceph-mon[35276]: pgmap v404: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:48.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:47 smithi164 ceph-mon[35327]: pgmap v405: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:48.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:47 smithi016 ceph-mon[31415]: pgmap v405: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:48.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:47 smithi069 ceph-mon[35276]: pgmap v405: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:49.171 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:13:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:13:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2049, 2024-05-27T07:13:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2051, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 615277, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 651583, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.416398639, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.130175263, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.093973989, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.146686108, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2049, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.338544112, 2024-05-27T07:13:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000165224 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1255, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1239, 2024-05-27T07:13:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1872490, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 482122, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.324318758, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.106720716, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.136137127, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.090184754, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1239, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.243446559, 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000196486 2024-05-27T07:13:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1648224, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 192992 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 233, 2024-05-27T07:13:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 588, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 545150, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.050579135, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.038574013, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.005765861, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000028417, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 230, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.061679328, 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000268170 2024-05-27T07:13:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 544832 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4764, 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.275901501, 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000057913 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-27T07:13:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-05-27T07:13:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-27T07:13:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:13:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-05-27T07:13:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-05-27T07:13:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 10, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-05-27T07:13:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4197372, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4197231, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 10 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-05-27T07:13:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 117888, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215440 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-27T07:13:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:13:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 5, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 3, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 4, 2024-05-27T07:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 4261, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 198961, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 56, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-27T07:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9368, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 178, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-27T07:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2959, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2959, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2959, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 5.953909094, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002012135 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-27T07:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2959, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2960 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2959, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2959, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1926, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1031, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-05-27T07:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-27T07:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 107, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 17, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-05-27T07:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 7, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 2, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3303, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2170604, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3303, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3303, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2170604, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2959, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 3073, 2024-05-27T07:13:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2959, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3073, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2959, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2959, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2959, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2959, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:13:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55d8d91549c0": { 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-27T07:13:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:49.221 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:13:50.246 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:49 smithi016 ceph-mon[31415]: pgmap v406: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:50.328 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-27T07:13:50.329 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T07:13:50.329 DEBUG:tasks.check_counter:Getting stats from d 2024-05-27T07:13:50.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-27T07:13:50.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:49 smithi164 ceph-mon[35327]: pgmap v406: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:50.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:49 smithi069 ceph-mon[35276]: pgmap v406: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:51.000 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:50 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-05-27T07:13:51.001 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:50 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:51.001 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:50 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.777s CPU time. 2024-05-27T07:13:51.001 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:50 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:13:51.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 podman[71605]: 2024-05-27 07:13:50.915958731 +0000 UTC m=+0.035832604 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:13:51.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 podman[71605]: 2024-05-27 07:13:51.045427444 +0000 UTC m=+0.165301344 container create 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T07:13:51.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 podman[71605]: 2024-05-27 07:13:51.402545974 +0000 UTC m=+0.522419921 container init 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T07:13:51.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 podman[71605]: 2024-05-27 07:13:51.412387628 +0000 UTC m=+0.532261511 container start 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-27T07:13:51.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 bash[71605]: 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 2024-05-27T07:13:51.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:13:51.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 ceph-mds[71624]: -- 172.21.15.164:0/1278038968 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb8fe73a40 con 0x55fb8febb600 2024-05-27T07:13:51.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: starting mds.c at 2024-05-27T07:13:52.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-mon[35327]: pgmap v407: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 8 op/s 2024-05-27T07:13:52.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-mon[35327]: daemon mds.c restarted 2024-05-27T07:13:52.259 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4cbc200700 time 2024-05-27T07:13:52.135665+0000 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4cd1d79ba6] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.260 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: clone() 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: *** Caught signal (Aborted) ** 2024-05-27T07:13:52.261 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: in thread 7f4cbc200700 thread_name:md_log_replay 2024-05-27T07:13:52.262 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.137+0000 7f4cbc200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4cbc200700 time 2024-05-27T07:13:52.135665+0000 2024-05-27T07:13:52.262 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:52.262 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.262 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4cd1d79ba6] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: clone() 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4cd08f6d20] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: gsignal() 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: abort() 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4cd1d79ced] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.264 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 19: clone() 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.140+0000 7f4cbc200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:52.265 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: in thread 7f4cbc200700 thread_name:md_log_replay 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4cd08f6d20] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: gsignal() 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: abort() 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4cd1d79ced] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.266 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 19: clone() 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: -1> 2024-05-27T07:13:52.137+0000 7f4cbc200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4cbc200700 time 2024-05-27T07:13:52.135665+0000 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.267 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4cd1d79ba6] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.268 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: clone() 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 0> 2024-05-27T07:13:52.140+0000 7f4cbc200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: in thread 7f4cbc200700 thread_name:md_log_replay 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4cd08f6d20] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: gsignal() 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: abort() 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4cd1d79ced] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.270 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 19: clone() 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: -1> 2024-05-27T07:13:52.137+0000 7f4cbc200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4cbc200700 time 2024-05-27T07:13:52.135665+0000 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4cd1d79ba6] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.271 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: clone() 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 0> 2024-05-27T07:13:52.140+0000 7f4cbc200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:13:52.272 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: in thread 7f4cbc200700 thread_name:md_log_replay 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4cd08f6d20] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2: gsignal() 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 3: abort() 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4cd1d79ced] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4cd1d79dd1] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fb6cd2723d] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fb6cd12971] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fb6cd10fa0] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fb6cd11651] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fb6cca6347] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fb6c9a287e] 2024-05-27T07:13:52.273 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fb6c9a7e94] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fb6c9b2969] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fb6cd4b48c] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fb6cd549cb] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 16: (Thread::entry_wrapper()+0x43) [0x7f4cd1d4872d] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4cd1d48749] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4cd08ec1ca] 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 19: clone() 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71620]: 2024-05-27T07:13:52.274 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 podman[71655]: 2024-05-27 07:13:52.268757086 +0000 UTC m=+0.033903657 container died 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:13:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:51 smithi016 ceph-mon[31415]: pgmap v407: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 8 op/s 2024-05-27T07:13:52.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:52 smithi016 ceph-mon[31415]: daemon mds.c restarted 2024-05-27T07:13:52.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:52 smithi069 ceph-mon[35276]: pgmap v407: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 8 op/s 2024-05-27T07:13:52.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:52 smithi069 ceph-mon[35276]: daemon mds.c restarted 2024-05-27T07:13:53.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:13:53.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/1320645011,v1:172.21.15.164:6837/1320645011] up:boot 2024-05-27T07:13:53.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:13:53.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:53.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:53.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:53 smithi016 ceph-mon[31415]: pgmap v409: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:53.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 podman[71655]: 2024-05-27 07:13:52.973096763 +0000 UTC m=+0.738243381 container remove 271b817a61e82b4991be04acd6862069065be04292629e135591c1b127ba8ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-27T07:13:53.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:52 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:13:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:13:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/1320645011,v1:172.21.15.164:6837/1320645011] up:boot 2024-05-27T07:13:53.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:13:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:53.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:53 smithi164 ceph-mon[35327]: pgmap v409: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/1320645011,v1:172.21.15.164:6837/1320645011] up:boot 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:13:53.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:53 smithi069 ceph-mon[35276]: pgmap v409: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.004201006, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T07:13:53.547 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001009008, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.548 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 5, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 8, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3152, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 80112, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.005290617, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000998616, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.004072254, 2024-05-27T07:13:53.549 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002482382, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000354626 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-27T07:13:53.550 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 80112 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 11, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 10, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 155328, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 4176, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.006493072, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000889643, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.008461677, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000177137, 2024-05-27T07:13:53.551 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 9, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002081124, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000231236 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 155328, 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 4176 2024-05-27T07:13:53.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:13:53.553 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:13:53.554 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:13:53.555 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:13:53.557 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148056, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 26, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-27T07:13:53.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 12, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 153847, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 12, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:53.560 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 12, 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 153847, 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:53.561 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:13:53.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:53 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:13:53.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:13:53 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.617s CPU time. 2024-05-27T07:13:54.513 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-27T07:13:54.513 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-27T07:13:54.513 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-27T07:13:54.514 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-27T07:13:54.514 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-27T07:13:54.514 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-27T07:13:54.514 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-27T07:13:54.514 DEBUG:tasks.check_counter:Getting stats from g 2024-05-27T07:13:54.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-27T07:13:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:55.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:55 smithi016 ceph-mon[31415]: pgmap v410: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:55.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:55 smithi164 ceph-mon[35327]: pgmap v410: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:55.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:55 smithi069 ceph-mon[35276]: pgmap v410: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:13:57.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:57 smithi016 ceph-mon[31415]: pgmap v411: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:57.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:57 smithi164 ceph-mon[35327]: pgmap v411: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:57.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:57 smithi069 ceph-mon[35276]: pgmap v411: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1012, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1014, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 339977, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 321817, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.264911387, 2024-05-27T07:13:57.767 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.107587521, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.056296769, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.076990145, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1012, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.190435663, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000188177 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.768 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1921, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2152, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 492787, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1158067, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.348263195, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.124754847, 2024-05-27T07:13:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.077757686, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.114530195, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2150, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.387841362, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000180391 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.770 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 549040 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 351, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 335, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1660708, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 195605, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.169510634, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.062532061, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.121919082, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.015739177, 2024-05-27T07:13:57.771 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 334, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.052936578, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000158492 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1659360, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194640 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.772 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4736, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.277698933, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000058635 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-27T07:13:57.773 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.774 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-05-27T07:13:57.775 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-05-27T07:13:57.776 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T07:13:57.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-27T07:13:57.778 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:13:57.779 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-27T07:13:57.780 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 7, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-27T07:13:57.781 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4196449, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4196308, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 7 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-05-27T07:13:57.782 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 117632, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215440 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-27T07:13:57.783 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.784 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-27T07:13:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-27T07:13:57.788 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.789 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.790 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.791 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.792 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.793 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.795 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:13:57.797 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-05-27T07:13:57.798 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 2, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 4271, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:13:57.799 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:13:57.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:13:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 207058, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 61, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-27T07:13:57.802 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9368, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 178, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-05-27T07:13:57.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2930, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2930, 2024-05-27T07:13:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2930, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 23.315644009, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.007957557 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2930, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2931 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2930, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2930, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-27T07:13:57.805 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1908, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1020, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-27T07:13:57.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-27T07:13:57.807 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-27T07:13:57.808 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 108, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 18, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 7, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 1, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.809 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-27T07:13:57.810 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3280, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2177510, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3280, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:57.811 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3280, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2177510, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2930, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2150, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2930, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2150, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-27T07:13:57.813 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2930, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2930, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2930, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2930, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.814 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:57.815 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x564988482ea0": { 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-27T07:13:57.816 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:13:57.817 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-27T07:13:58.838 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T07:13:58.838 DEBUG:tasks.check_counter:Getting stats from j 2024-05-27T07:13:58.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-27T07:13:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:13:59.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:13:59 smithi016 ceph-mon[31415]: pgmap v412: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:59.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:13:59 smithi164 ceph-mon[35327]: pgmap v412: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:13:59.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:13:59 smithi069 ceph-mon[35276]: pgmap v412: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:14:01.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:01 smithi016 ceph-mon[31415]: pgmap v413: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:14:01.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:01 smithi164 ceph-mon[35327]: pgmap v413: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:14:01.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:01 smithi069 ceph-mon[35276]: pgmap v413: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 8 op/s 2024-05-27T07:14:02.139 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 5, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2992, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.006283101, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000091812, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002116097, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T07:14:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002356843, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000785614 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 243, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 202160, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.055985255, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.042694687, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.006960708, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T07:14:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 239, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.049719174, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000208030 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 202160 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:14:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 330, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 319, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1743360, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 196400, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.143174120, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.050951859, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.101038954, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.014568256, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 318, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.080736238, 2024-05-27T07:14:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000253887 2024-05-27T07:14:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:14:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:14:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1743360, 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196400 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-27T07:14:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:14:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:14:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149011, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-05-27T07:14:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-27T07:14:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 332, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1696778, 2024-05-27T07:14:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 332, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 332, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1696778, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-27T07:14:02.152 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-27T07:14:03.096 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-27T07:14:03.096 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-27T07:14:03.096 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-27T07:14:03.097 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-27T07:14:03.097 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-27T07:14:03.097 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-27T07:14:03.097 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-27T07:14:03.097 DEBUG:tasks.check_counter:Getting stats from b 2024-05-27T07:14:03.097 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-27T07:14:03.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:03 smithi016 ceph-mon[31415]: pgmap v414: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-27T07:14:03.539 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:14:03.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:03 smithi164 ceph-mon[35327]: pgmap v414: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-27T07:14:03.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:03 smithi069 ceph-mon[35276]: pgmap v414: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-27T07:14:03.982 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:03 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-05-27T07:14:03.982 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:03 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:03.982 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:03 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.617s CPU time. 2024-05-27T07:14:03.982 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:03 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:04.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:03 smithi164 podman[71749]: 2024-05-27 07:14:03.896358736 +0000 UTC m=+0.033443450 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:14:04.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 podman[71749]: 2024-05-27 07:14:04.019268432 +0000 UTC m=+0.156353173 container create 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:14:04.269 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 podman[71749]: 2024-05-27 07:14:04.278042878 +0000 UTC m=+0.415127692 container init 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=) 2024-05-27T07:14:04.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 podman[71749]: 2024-05-27 07:14:04.288809291 +0000 UTC m=+0.425894014 container start 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T07:14:04.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 ceph-mds[71767]: -- 172.21.15.164:0/4016840090 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e57ac25a40 con 0x55e57ac6da80 2024-05-27T07:14:04.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: starting mds.c at 2024-05-27T07:14:04.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 bash[71749]: 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 2024-05-27T07:14:04.582 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:04 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:05.480 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:05 smithi069 ceph-mon[35276]: pgmap v415: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-27T07:14:05.481 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:05 smithi069 ceph-mon[35276]: daemon mds.c restarted 2024-05-27T07:14:05.482 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-mon[35327]: pgmap v415: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-27T07:14:05.483 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-mon[35327]: daemon mds.c restarted 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fb725800700 time 2024-05-27T07:14:05.348326+0000 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb73b3e9ba6] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.484 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: clone() 2024-05-27T07:14:05.485 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.350+0000 7fb725800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fb725800700 time 2024-05-27T07:14:05.348326+0000 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb73b3e9ba6] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.486 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: clone() 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: *** Caught signal (Aborted) ** 2024-05-27T07:14:05.487 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: in thread 7fb725800700 thread_name:md_log_replay 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb739f66d20] 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: gsignal() 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: abort() 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fb73b3e9ced] 2024-05-27T07:14:05.488 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.489 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 17: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 19: clone() 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.353+0000 7fb725800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: in thread 7fb725800700 thread_name:md_log_replay 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.490 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb739f66d20] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: gsignal() 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: abort() 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fb73b3e9ced] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.491 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 17: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 19: clone() 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: -1> 2024-05-27T07:14:05.350+0000 7fb725800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fb725800700 time 2024-05-27T07:14:05.348326+0000 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb73b3e9ba6] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.492 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: clone() 2024-05-27T07:14:05.493 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 0> 2024-05-27T07:14:05.353+0000 7fb725800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: in thread 7fb725800700 thread_name:md_log_replay 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb739f66d20] 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: gsignal() 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: abort() 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fb73b3e9ced] 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.494 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 17: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 19: clone() 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:05.495 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: -1> 2024-05-27T07:14:05.350+0000 7fb725800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fb725800700 time 2024-05-27T07:14:05.348326+0000 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb73b3e9ba6] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.496 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: clone() 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 0> 2024-05-27T07:14:05.353+0000 7fb725800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: in thread 7fb725800700 thread_name:md_log_replay 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb739f66d20] 2024-05-27T07:14:05.497 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2: gsignal() 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 3: abort() 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fb73b3e9ced] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fb73b3e9dd1] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e5762e323d] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e5762ce971] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e5762ccfa0] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e5762cd651] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e576262347] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e575f5e87e] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e575f63e94] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e575f6e969] 2024-05-27T07:14:05.498 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e57630748c] 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e5763109cb] 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 16: (Thread::entry_wrapper()+0x43) [0x7fb73b3b872d] 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 17: (Thread::_entry_func(void*)+0xd) [0x7fb73b3b8749] 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fb739f5c1ca] 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 19: clone() 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[71763]: 2024-05-27T07:14:05.499 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:05 smithi164 podman[71798]: 2024-05-27 07:14:05.492564795 +0000 UTC m=+0.032039041 container died 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:14:05.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:05 smithi016 ceph-mon[31415]: pgmap v415: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-27T07:14:05.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:05 smithi016 ceph-mon[31415]: daemon mds.c restarted 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2876, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2951, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 928488, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 5158550, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-05-27T07:14:06.026 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.677071889, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.278186037, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.221259205, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.134339306, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2956, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.882739016, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000298626 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2856, 2024-05-27T07:14:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001586484, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000555 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 343888 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3253, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2680, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1238874, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1217572, 2024-05-27T07:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 11, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.716471168, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.261339551, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.260650192, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.173722111, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2681, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.129567638, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000421323 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.029 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3218, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001547251, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000480 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 43632, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 178624 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3740, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2724, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3015750, 2024-05-27T07:14:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1963159, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 88, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 11, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.928102786, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.322291621, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.432548702, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.185359106, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2713, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.630728604, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000232483 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2998, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001568710, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000523 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2259824, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 370592 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.032 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 877, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.717419855, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000818038 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2509, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.145184425, 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000057865 2024-05-27T07:14:06.033 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 273, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 273, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 273, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.988254292, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003619979 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-27T07:14:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 21, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 1, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 162, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 60, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 102, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 75, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 0, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 3, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 3, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 2, 2024-05-27T07:14:06.035 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 359, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 341, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 308, 2024-05-27T07:14:06.036 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 4, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 234, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 308, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 114, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 11, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 103, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 194, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 87, 2024-05-27T07:14:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 185, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 275, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 194, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 4, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 83 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 19, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-05-27T07:14:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 40, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 4, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 22, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 18, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:14:06.039 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 11, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 1, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 524, 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 524, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 2, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-27T07:14:06.041 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 7994043, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 271, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.937248707, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.007148519 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 135, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 164, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 29, 2024-05-27T07:14:06.042 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 29, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 30, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 1, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 162, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 180, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 18, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 3, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 40, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 37, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 125868, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 231824 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-27T07:14:06.043 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 368, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 1, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 273, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 43, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 2, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 22, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.060730749, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002760488 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 156, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.085874215, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000550475 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003011951, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000752987 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.045 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 13, 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.006727760, 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000517520 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 10, 2024-05-27T07:14:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.044214469, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004421446 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 13, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.170628219, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.013125247 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001193614, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001193614 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 12, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.055597237, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004633103 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.011492440, 2024-05-27T07:14:06.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005746220 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.158682845, 2024-05-27T07:14:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.039670711 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 7, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.228816756, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.032688108 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:06.052 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003176167, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001588083 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 9, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.069243687, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.007693743 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.054 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 18, 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.088864183, 2024-05-27T07:14:06.055 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004936899 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.057 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 7, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 5, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 54, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 27, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 758, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.058 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:14:06.059 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:14:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 333638, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 133, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-27T07:14:06.061 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9368, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 178, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 528038, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2978, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-27T07:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 3827, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 3939260, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 37, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 3827, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3827, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 165.293413454, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.043191380 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.063 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3827, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 3896 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 3827, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 332, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 3495, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 11, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 13, 2024-05-27T07:14:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 273, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 49, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 1, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 3108, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 319, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 22, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:14:06.065 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-27T07:14:06.066 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 109, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 30, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2024-05-27T07:14:06.067 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 81, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 19 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 1 2024-05-27T07:14:06.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 522, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 5, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 2596, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 18, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 9865, 2024-05-27T07:14:06.069 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 4209777, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 9865, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 9865, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 4209777, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 3827, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 3942086, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 3827, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 3942086, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.071 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 3827, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 3827, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 3827, 2024-05-27T07:14:06.072 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 3827, 2024-05-27T07:14:06.073 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:06.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.073 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 18, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 2826, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 18, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2826, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x556f13b26b60": { 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 60, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 524, 2024-05-27T07:14:06.075 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 3799739, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 524, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 271, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 3799679, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-27T07:14:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:06 smithi069 ceph-mon[35276]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:14:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:06 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/211994695,v1:172.21.15.164:6837/211994695] up:boot 2024-05-27T07:14:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:06 smithi069 ceph-mon[35276]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:06 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:06.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:06 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:06.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:06 smithi164 podman[71798]: 2024-05-27 07:14:06.29283659 +0000 UTC m=+0.832310845 container remove 2690cb1c1c5c8d3910419c602428628c05aaa33ef625fab8f27e1465ff6da704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T07:14:06.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:06 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:14:06.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:06 smithi164 ceph-mon[35327]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:14:06.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:06 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/211994695,v1:172.21.15.164:6837/211994695] up:boot 2024-05-27T07:14:06.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:06 smithi164 ceph-mon[35327]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:06.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:06 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:06.582 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:06 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:06.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:06 smithi016 ceph-mon[31415]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:14:06.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:06 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/211994695,v1:172.21.15.164:6837/211994695] up:boot 2024-05-27T07:14:06.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:06 smithi016 ceph-mon[31415]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:06.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:06 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:06.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:06 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:07.088 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-27T07:14:07.088 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-27T07:14:07.088 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-05-27T07:14:07.088 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-27T07:14:07.089 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-27T07:14:07.089 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=13 2024-05-27T07:14:07.089 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=7 2024-05-27T07:14:07.089 DEBUG:tasks.check_counter:Getting stats from e 2024-05-27T07:14:07.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-27T07:14:07.233 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:06 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:14:07.233 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:06 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.570s CPU time. 2024-05-27T07:14:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:07 smithi069 ceph-mon[35276]: pgmap v417: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-27T07:14:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:07 smithi069 ceph-mon[35276]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:14:07.368 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:07 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:14:07.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:07 smithi164 ceph-mon[35327]: pgmap v417: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-27T07:14:07.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:07 smithi164 ceph-mon[35327]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:14:07.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:07 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:14:07.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:07 smithi016 ceph-mon[31415]: pgmap v417: 129 pgs: 129 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-27T07:14:07.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:07 smithi016 ceph-mon[31415]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:14:07.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:07 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:14:07.995 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:14:09.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:09 smithi164 ceph-mon[35327]: pgmap v419: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-05-27T07:14:09.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:09 smithi016 ceph-mon[31415]: pgmap v419: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-05-27T07:14:09.637 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:09 smithi069 ceph-mon[35276]: pgmap v419: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-05-27T07:14:10.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:10 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:14:10.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:10 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:14:10.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:10 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:14:10.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:10 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:14:10.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:10 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:14:10.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:10 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:14:10.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:10 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:14:10.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:10 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:14:10.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:10 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' 2024-05-27T07:14:11.340 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:11 smithi069 ceph-mon[35276]: pgmap v420: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 34 KiB/s wr, 28 op/s 2024-05-27T07:14:11.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:11 smithi164 ceph-mon[35327]: pgmap v420: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 34 KiB/s wr, 28 op/s 2024-05-27T07:14:11.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:11 smithi016 ceph-mon[31415]: pgmap v420: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 34 KiB/s wr, 28 op/s 2024-05-27T07:14:11.905 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 454, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 509, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1716917, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 292344, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 14, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.191848728, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.054829859, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.153836312, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.015795628, 2024-05-27T07:14:11.906 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 508, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.096566689, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000190091 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 128, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000062997, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000492 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1709584, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 221744 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.907 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 46, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 289, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 9421, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 772926, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 31, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 8, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.062845285, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.023803391, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.030623039, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000484373, 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 286, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.068865563, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000240788 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 37, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000011795, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000318 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2160, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 493728 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.909 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 749, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 786, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 254784, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 434359, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.252376231, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.140023793, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.110460995, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010834079, 2024-05-27T07:14:11.910 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 785, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.121827932, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000155194 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1471, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000782252, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000531 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 1424 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.911 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 68, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003432219, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000050473 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000010769, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000005384 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 11, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 7, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 7, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.004059290, 2024-05-27T07:14:11.913 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000579898 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 4, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 11, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 16, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 16, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 13, 2024-05-27T07:14:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 8, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 2, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 2, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 3, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 25, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 17, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 1, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 5, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-05-27T07:14:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 22, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 10738143856, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:14:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 1 2024-05-27T07:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 12, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 8, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 10, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-27T07:14:11.918 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-27T07:14:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 10, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 10, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 2, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-05-27T07:14:11.920 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4197372, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 9, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.051416467, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005712940 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 20, 2024-05-27T07:14:11.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 24, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 4, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 19, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 32, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 13, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 16, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 24, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 8, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 2, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 5, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 3, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 120704, 2024-05-27T07:14:11.922 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223632 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 12, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 11, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 33, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-27T07:14:11.923 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003574050, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000595675 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-27T07:14:11.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000485240, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000485240 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.925 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.927 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.929 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-27T07:14:11.930 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.933 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-27T07:14:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 5, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 3, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 4, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 1, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 762, 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-27T07:14:11.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:14:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:14:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 274710, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 110, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 88, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 1, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-27T07:14:11.938 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9848, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 190, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 84213, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 336, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-27T07:14:11.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 48, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 13644, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 33, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 48, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 48, 2024-05-27T07:14:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.778982588, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.016228803 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 48, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 71 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 48, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 0, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 48, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-05-27T07:14:11.941 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 9, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 9, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 6, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-27T07:14:11.942 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-27T07:14:11.943 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-27T07:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 110, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 20, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 8, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 10, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 2, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 16, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 1 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-27T07:14:11.945 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-05-27T07:14:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 1245, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1844533, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 1245, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1245, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1844533, 2024-05-27T07:14:11.947 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:11.948 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 48, 2024-05-27T07:14:11.949 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 13644, 2024-05-27T07:14:11.949 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 48, 2024-05-27T07:14:11.949 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 13644, 2024-05-27T07:14:11.949 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:11.950 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 48, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 48, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 48, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 48, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:11.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.952 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x563010b2ca90": { 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 10, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 3068, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 10, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 9, 2024-05-27T07:14:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 3068, 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:11.954 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=12 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-27T07:14:13.431 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T07:14:13.432 DEBUG:tasks.check_counter:Getting stats from h 2024-05-27T07:14:13.432 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-27T07:14:13.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:13 smithi164 ceph-mon[35327]: pgmap v421: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-27T07:14:13.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:13 smithi016 ceph-mon[31415]: pgmap v421: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-27T07:14:13.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:13 smithi069 ceph-mon[35276]: pgmap v421: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-27T07:14:14.135 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:14:15.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:15 smithi164 ceph-mon[35327]: pgmap v422: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:14:15.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:15 smithi016 ceph-mon[31415]: pgmap v422: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:14:15.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:15 smithi069 ceph-mon[35276]: pgmap v422: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 756, 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 961, 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 255748, 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 648417, 2024-05-27T07:14:16.413 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.285455276, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.157705934, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.118278533, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010248160, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 960, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.163491531, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000170303 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1473, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000724730, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000492 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 344576 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 51, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 170, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 9617, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 433209, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 34, 2024-05-27T07:14:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.057383342, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.015152348, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.036320781, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000263401, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 167, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.098502873, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000589837 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 86, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000039277, 2024-05-27T07:14:16.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000456 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 148976 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 416, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 390, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1712393, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 263958, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 10, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.151164590, 2024-05-27T07:14:16.417 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.045709243, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.122801156, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.014503041, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 389, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.057672399, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000148258 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 12, 2024-05-27T07:14:16.418 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000003788, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000315 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1709296, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 205904 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 57, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.002726305, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000047829 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000019424, 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000009712 2024-05-27T07:14:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 2, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 2, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.005110222, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002555111 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-27T07:14:16.421 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 11, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 16, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 16, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 13, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 8, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 2, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 2, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 3, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 12, 2024-05-27T07:14:16.422 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 8, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 4, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-05-27T07:14:16.423 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 22, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 10738143856, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 1, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-05-27T07:14:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 1, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 14, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 9, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 10, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-05-27T07:14:16.425 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T07:14:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 7, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 7, 2024-05-27T07:14:16.427 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 2, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4196449, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.026991804, 2024-05-27T07:14:16.428 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004498634 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 20, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 24, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 4, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 19, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 33, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 14, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 16, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 24, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 8, 2024-05-27T07:14:16.429 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 2, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 2, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 120576, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223632 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 5, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 2, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 19, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 1, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.430 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.005110222, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002555111 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-27T07:14:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-27T07:14:16.433 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.433 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.433 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.433 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.434 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.436 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-27T07:14:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.439 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.440 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-27T07:14:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-27T07:14:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 3, 2024-05-27T07:14:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 2, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 1, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 1, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 805, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:14:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:14:16.446 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-27T07:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 280177, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 93, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:14:16.448 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9848, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 190, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 84229, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 338, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 43, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 12602, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 32, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 43, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 43, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.530988116, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.012348560 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-27T07:14:16.450 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 43, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 66 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 43, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 0, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 43, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-27T07:14:16.451 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 6, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 7, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 6, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 0, 2024-05-27T07:14:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-27T07:14:16.453 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-27T07:14:16.454 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 110, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 21, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 9, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 11, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 2, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 16, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 1 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-27T07:14:16.455 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.456 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 1219, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1844565, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 1219, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1219, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1844565, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 43, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 12602, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 43, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 12602, 2024-05-27T07:14:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:16.459 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 43, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 43, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 43, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 43, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x5580920469c0": { 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 7, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 2145, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 7, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2145, 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:16.462 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:16.463 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:16.463 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:16.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-27T07:14:17.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:17 smithi164 ceph-mon[35327]: pgmap v423: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-27T07:14:17.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:17 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-05-27T07:14:17.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:17 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:17.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:17 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.570s CPU time. 2024-05-27T07:14:17.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:17 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:17.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:17 smithi016 ceph-mon[31415]: pgmap v423: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-27T07:14:17.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:17 smithi069 ceph-mon[35276]: pgmap v423: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-27T07:14:17.851 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-27T07:14:17.852 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T07:14:17.852 DEBUG:tasks.check_counter:Getting stats from k 2024-05-27T07:14:17.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-27T07:14:18.315 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:17 smithi164 podman[72016]: 2024-05-27 07:14:17.844321577 +0000 UTC m=+0.035152215 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:14:18.315 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 podman[72016]: 2024-05-27 07:14:18.018299611 +0000 UTC m=+0.209130286 container create d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:14:18.317 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:14:18.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 podman[72016]: 2024-05-27 07:14:18.455365954 +0000 UTC m=+0.646196652 container init d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T07:14:18.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 podman[72016]: 2024-05-27 07:14:18.465531501 +0000 UTC m=+0.656362210 container start d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:14:18.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 ceph-mds[72034]: -- 172.21.15.164:0/2852301483 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560be2ef1a40 con 0x560be2f39600 2024-05-27T07:14:18.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: starting mds.c at 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 bash[72016]: d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:18 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f734bc00700 time 2024-05-27T07:14:19.008518+0000 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f73616c9ba6] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.135 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.136 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: clone() 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.009+0000 7f734bc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f734bc00700 time 2024-05-27T07:14:19.008518+0000 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f73616c9ba6] 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.137 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: clone() 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.138 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: *** Caught signal (Aborted) ** 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: in thread 7f734bc00700 thread_name:md_log_replay 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7360246d20] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: gsignal() 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: abort() 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f73616c9ced] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.139 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 19: clone() 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.012+0000 7f734bc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: in thread 7f734bc00700 thread_name:md_log_replay 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7360246d20] 2024-05-27T07:14:19.140 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: gsignal() 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: abort() 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f73616c9ced] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.141 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 19: clone() 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: -1> 2024-05-27T07:14:19.009+0000 7f734bc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f734bc00700 time 2024-05-27T07:14:19.008518+0000 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f73616c9ba6] 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.142 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.143 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: clone() 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 0> 2024-05-27T07:14:19.012+0000 7f734bc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: in thread 7f734bc00700 thread_name:md_log_replay 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7360246d20] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: gsignal() 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: abort() 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f73616c9ced] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.144 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.145 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 19: clone() 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: -1> 2024-05-27T07:14:19.009+0000 7f734bc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f734bc00700 time 2024-05-27T07:14:19.008518+0000 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f73616c9ba6] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.146 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: clone() 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 0> 2024-05-27T07:14:19.012+0000 7f734bc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: in thread 7f734bc00700 thread_name:md_log_replay 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:19.147 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7360246d20] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2: gsignal() 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 3: abort() 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f73616c9ced] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f73616c9dd1] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560bcd25e23d] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560bcd249971] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 8: (SnapRealm::check_cache() const+0x13e) [0x560bcd247fa0] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560bcd248651] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560bcd1dd347] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560bcced987e] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560bccedee94] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560bccee9969] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 14: (MDLog::_replay_thread()+0x1f38) [0x560bcd28248c] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560bcd28b9cb] 2024-05-27T07:14:19.148 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 16: (Thread::entry_wrapper()+0x43) [0x7f736169872d] 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7361698749] 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f736023c1ca] 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 19: clone() 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72030]: 2024-05-27T07:14:19.149 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 podman[72065]: 2024-05-27 07:14:19.144063724 +0000 UTC m=+0.033820556 container died d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T07:14:19.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: pgmap v424: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-27T07:14:19.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: daemon mds.c restarted 2024-05-27T07:14:19.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:14:19.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/3230038190,v1:172.21.15.164:6837/3230038190] up:boot 2024-05-27T07:14:19.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:19.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:19 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:19.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: pgmap v424: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-27T07:14:19.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: daemon mds.c restarted 2024-05-27T07:14:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:14:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/3230038190,v1:172.21.15.164:6837/3230038190] up:boot 2024-05-27T07:14:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:19.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:19 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: pgmap v424: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: daemon mds.c restarted 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/3230038190,v1:172.21.15.164:6837/3230038190] up:boot 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:19.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:19 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:20.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 podman[72065]: 2024-05-27 07:14:19.973193859 +0000 UTC m=+0.862950721 container remove d2c695833b8298bc0767f69adb6afa7d1b9f902629efccf1ddfa7bd4649c57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T07:14:20.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:19 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 455, 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 471, 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1718713, 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 283694, 2024-05-27T07:14:20.738 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 14, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.185482830, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.051047682, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.145604662, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.018276280, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 470, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.119379342, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000253998 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:20.739 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 97, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000045039, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000464 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1710928, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 217600 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 40, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 285, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 10132, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 753031, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 30, 2024-05-27T07:14:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.060895332, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.022795762, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.030370209, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000338099, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 283, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.091169354, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000322153 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 26, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000008411, 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000323 2024-05-27T07:14:20.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 529200 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 747, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 793, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 252524, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 492452, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 8, 2024-05-27T07:14:20.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.252114013, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.135708648, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.114993268, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.011173073, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 791, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.159557187, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000201715 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1480, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000689027, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000465 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-27T07:14:20.743 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 976 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 67, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003859852, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000057609 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000012006, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000006003 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 1, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 1, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 11, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 16, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 16, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 13, 2024-05-27T07:14:20.746 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 8, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 3, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 10, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 5, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 1, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 4, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-27T07:14:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 22, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 10738143856, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T07:14:20.748 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T07:14:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 14, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 9, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 10, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-27T07:14:20.750 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-27T07:14:20.751 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 10, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 10, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 2, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-27T07:14:20.752 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4197372, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 9, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.042190335, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004687815 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-27T07:14:20.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 20, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 24, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 4, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 19, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 33, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 14, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 16, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 24, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 8, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2024-05-27T07:14:20.754 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 120064, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223632 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 1, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 1, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 34, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.758 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-27T07:14:20.758 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.758 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.758 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.765 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.766 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.767 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.768 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 5, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 3, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 773, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-27T07:14:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T07:14:20.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-27T07:14:20.771 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 272483, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 95, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-27T07:14:20.772 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9848, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 190, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 83453, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 329, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-27T07:14:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 47, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 13550, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 33, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 47, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 47, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.621986102, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.013233746 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.774 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 47, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 69 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 47, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 47, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 0, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 9, 2024-05-27T07:14:20.775 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 9, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 6, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-27T07:14:20.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-27T07:14:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 111, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 21, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 8, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 13, 2024-05-27T07:14:20.778 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 5, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 14, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 1 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-05-27T07:14:20.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:20.780 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 1238, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1845563, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 1238, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1238, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1845563, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.781 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 47, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 13550, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 47, 2024-05-27T07:14:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 13550, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:20.783 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 47, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 47, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 47, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 47, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.784 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x55d4998849c0": { 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 10, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 3068, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 10, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-27T07:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 9, 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 3068, 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-27T07:14:20.787 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-27T07:14:20.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:20 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:14:20.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:20 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.676s CPU time. 2024-05-27T07:14:21.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:21 smithi164 ceph-mon[35327]: pgmap v426: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:21.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:21 smithi016 ceph-mon[31415]: pgmap v426: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:21.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:21 smithi069 ceph-mon[35276]: pgmap v426: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:21.712 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=14 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-27T07:14:21.713 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T07:14:21.713 DEBUG:tasks.check_counter:Getting stats from c 2024-05-27T07:14:21.713 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-27T07:14:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:14:23.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:23 smithi164 ceph-mon[35327]: pgmap v427: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:23.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:23 smithi016 ceph-mon[31415]: pgmap v427: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:23.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:23 smithi069 ceph-mon[35276]: pgmap v427: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:25.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:24 smithi164 ceph-mon[35327]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:14:25.324 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-05-27T07:14:25.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:24 smithi016 ceph-mon[31415]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:14:25.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:24 smithi069 ceph-mon[35276]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:14:26.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:25 smithi164 ceph-mon[35327]: pgmap v429: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 8 op/s 2024-05-27T07:14:26.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-27T07:14:26.295 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump' 2024-05-27T07:14:26.297 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-27T07:14:26.308 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-27T07:14:26.308 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T07:14:26.308 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:14:26.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:25 smithi016 ceph-mon[31415]: pgmap v429: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 8 op/s 2024-05-27T07:14:26.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:25 smithi069 ceph-mon[35276]: pgmap v429: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 8 op/s 2024-05-27T07:14:26.380 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T07:14:26.380 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:14:26.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:14:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:14:28.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:27 smithi016 ceph-mon[31415]: pgmap v430: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 9 op/s 2024-05-27T07:14:28.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:27 smithi164 ceph-mon[35327]: pgmap v430: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 9 op/s 2024-05-27T07:14:28.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:27 smithi069 ceph-mon[35276]: pgmap v430: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 9 op/s 2024-05-27T07:14:29.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 -- 172.21.15.16:0/354606486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244110960 msgr2=0x7f8244110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:29.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 --2- 172.21.15.16:0/354606486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244110960 0x7f8244110d70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8234009990 tx=0x7f8234009ca0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 -- 172.21.15.16:0/354606486 shutdown_connections 2024-05-27T07:14:29.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 --2- 172.21.15.16:0/354606486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f8244117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 --2- 172.21.15.16:0/354606486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244110960 0x7f8244110d70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 --2- 172.21.15.16:0/354606486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f824410a980 0x7f824410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 -- 172.21.15.16:0/354606486 >> 172.21.15.16:0/354606486 conn(0x7f8244077c70 msgr2=0x7f82440780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.766+0000 7f824b000700 1 -- 172.21.15.16:0/354606486 shutdown_connections 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.767+0000 7f824b000700 1 -- 172.21.15.16:0/354606486 wait complete. 2024-05-27T07:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.768+0000 7f824b000700 1 Processor -- start 2024-05-27T07:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.768+0000 7f824b000700 1 -- start start 2024-05-27T07:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f82441a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 0x7f82441a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f824411a910 con 0x7f824410a980 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f824411a790 con 0x7f8244110960 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f824b000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f824411aa90 con 0x7f824410b2d0 2024-05-27T07:14:29.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f8243e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f8243e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44510/0 (socket says 172.21.15.16:44510) 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f8243e00700 1 -- 172.21.15.16:0/1133880031 learned_addr learned my addr 172.21.15.16:0/1133880031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f8243400700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f82441a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.769+0000 7f8249000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 0x7f82441a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 -- 172.21.15.16:0/1133880031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 msgr2=0x7f82441a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f82441a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 -- 172.21.15.16:0/1133880031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 msgr2=0x7f82441a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 0x7f82441a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 -- 172.21.15.16:0/1133880031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441a6230 con 0x7f824410a980 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243400700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f82441a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8249000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 0x7f82441a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.770+0000 7f8243e00700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8234000c00 tx=0x7f8234005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:14:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.771+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234027070 con 0x7f824410a980 2024-05-27T07:14:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.771+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823400fd40 con 0x7f824410a980 2024-05-27T07:14:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.771+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441a64c0 con 0x7f824410a980 2024-05-27T07:14:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.771+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234022ca0 con 0x7f824410a980 2024-05-27T07:14:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.771+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a69a0 con 0x7f824410a980 2024-05-27T07:14:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.772+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82440512b0 con 0x7f824410a980 2024-05-27T07:14:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.775+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f823402a070 con 0x7f824410a980 2024-05-27T07:14:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.776+0000 7f8240c00700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 0x7f82280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.777+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9525+0+0 (secure 0 0 0) 0x7f82340e3c80 con 0x7f824410a980 2024-05-27T07:14:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.777+0000 7f8243400700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 0x7f82280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.778+0000 7f8243400700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 0x7f82280b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8230005950 tx=0x7f82300058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:14:29.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:29.779+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f823405fdf0 con 0x7f824410a980 2024-05-27T07:14:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.038+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f824402e5e0 con 0x7f824410a980 2024-05-27T07:14:30.029 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:29 smithi016 ceph-mon[31415]: pgmap v431: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 9 op/s 2024-05-27T07:14:30.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.039+0000 7f8240c00700 1 -- 172.21.15.16:0/1133880031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v112) ==== 81+0+944 (secure 0 0 0) 0x7f8234013070 con 0x7f824410a980 2024-05-27T07:14:30.029 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/702672487 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2755801148 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3562581452 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1801942445 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2589421123 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/454643589 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3708593763 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3327639895 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3317326035 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3906409731 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4243270602 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2755801148 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:30.031 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:30.031 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:30.031 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:30.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 msgr2=0x7f82280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:30.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 0x7f82280b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8230005950 tx=0x7f82300058e0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 msgr2=0x7f82441a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:30.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8234000c00 tx=0x7f8234005ef0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 shutdown_connections 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f82280b29f0 0x7f82280b4f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8244110960 0x7f82441a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824410a980 0x7f82441a1360 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 --2- 172.21.15.16:0/1133880031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824410b2d0 0x7f82441a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.044+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 >> 172.21.15.16:0/1133880031 conn(0x7f8244077c70 msgr2=0x7f824407bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.045+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 shutdown_connections 2024-05-27T07:14:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:30.045+0000 7f824b000700 1 -- 172.21.15.16:0/1133880031 wait complete. 2024-05-27T07:14:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:listed 16 entries 2024-05-27T07:14:30.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:29 smithi164 ceph-mon[35327]: pgmap v431: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 9 op/s 2024-05-27T07:14:30.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:29 smithi069 ceph-mon[35276]: pgmap v431: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 9 op/s 2024-05-27T07:14:30.924 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:30 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-05-27T07:14:30.925 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:30 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:30.925 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:30 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.676s CPU time. 2024-05-27T07:14:30.925 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:30 smithi164 systemd[1]: Starting Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:31.037 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-27T07:14:31.038 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:14:31.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:14:31.270 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:30 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1133880031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:31.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:30 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1133880031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:31.321 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 podman[72309]: 2024-05-27 07:14:30.968957014 +0000 UTC m=+0.035795098 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:14:31.321 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 podman[72309]: 2024-05-27 07:14:31.17377826 +0000 UTC m=+0.240616362 container create 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T07:14:31.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:30 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1133880031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:14:31.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 podman[72309]: 2024-05-27 07:14:31.502858301 +0000 UTC m=+0.569696405 container init 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T07:14:31.581 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 podman[72309]: 2024-05-27 07:14:31.513304824 +0000 UTC m=+0.580142955 container start 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T07:14:31.947 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 bash[72309]: 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b 2024-05-27T07:14:31.947 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 systemd[1]: Started Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:31.947 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 ceph-mds[72328]: -- 172.21.15.164:0/3981127859 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c72a0eda40 con 0x55c72a135180 2024-05-27T07:14:31.947 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: starting mds.c at 2024-05-27T07:14:32.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:31 smithi016 ceph-mon[31415]: pgmap v432: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:32.273 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:31 smithi016 ceph-mon[31415]: daemon mds.c restarted 2024-05-27T07:14:32.293 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:31 smithi069 ceph-mon[35276]: pgmap v432: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:32.293 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:31 smithi069 ceph-mon[35276]: daemon mds.c restarted 2024-05-27T07:14:32.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:31 smithi164 ceph-mon[35327]: pgmap v432: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 8 op/s 2024-05-27T07:14:32.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:31 smithi164 ceph-mon[35327]: daemon mds.c restarted 2024-05-27T07:14:32.332 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1282e00700 time 2024-05-27T07:14:32.082352+0000 2024-05-27T07:14:32.332 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:32.332 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.332 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f129889dba6] 2024-05-27T07:14:32.332 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.333 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: clone() 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.083+0000 7f1282e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1282e00700 time 2024-05-27T07:14:32.082352+0000 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f129889dba6] 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.334 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: clone() 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.335 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: *** Caught signal (Aborted) ** 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: in thread 7f1282e00700 thread_name:md_log_replay 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f129741ad20] 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: gsignal() 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: abort() 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f129889dced] 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.336 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 17: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 19: clone() 2024-05-27T07:14:32.337 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.086+0000 7f1282e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: in thread 7f1282e00700 thread_name:md_log_replay 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f129741ad20] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: gsignal() 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: abort() 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f129889dced] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.338 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.339 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 17: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 19: clone() 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: -1> 2024-05-27T07:14:32.083+0000 7f1282e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1282e00700 time 2024-05-27T07:14:32.082352+0000 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f129889dba6] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.340 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: clone() 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.341 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 0> 2024-05-27T07:14:32.086+0000 7f1282e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: in thread 7f1282e00700 thread_name:md_log_replay 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f129741ad20] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: gsignal() 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: abort() 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f129889dced] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.342 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.343 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 17: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 19: clone() 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: -1> 2024-05-27T07:14:32.083+0000 7f1282e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1282e00700 time 2024-05-27T07:14:32.082352+0000 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f129889dba6] 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.344 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: clone() 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.345 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 0> 2024-05-27T07:14:32.086+0000 7f1282e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: in thread 7f1282e00700 thread_name:md_log_replay 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f129741ad20] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2: gsignal() 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 3: abort() 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f129889dced] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f129889ddd1] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c71090f23d] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c7108fa971] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c7108f8fa0] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c7108f9651] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c71088e347] 2024-05-27T07:14:32.346 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c71058a87e] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c71058fe94] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c71059a969] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c71093348c] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c71093c9cb] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 16: (Thread::entry_wrapper()+0x43) [0x7f129886c72d] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 17: (Thread::_entry_func(void*)+0xd) [0x7f129886c749] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f12974101ca] 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 19: clone() 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c[72324]: 2024-05-27T07:14:32.347 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:32 smithi164 podman[72359]: 2024-05-27 07:14:32.20501074 +0000 UTC m=+0.035099020 container died 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T07:14:33.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:32 smithi069 ceph-mon[35276]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:14:33.117 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:32 smithi069 ceph-mon[35276]: mds.? [v2:172.21.15.164:6836/1142577163,v1:172.21.15.164:6837/1142577163] up:boot 2024-05-27T07:14:33.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:32 smithi069 ceph-mon[35276]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:33.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:32 smithi069 ceph-mon[35276]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:33.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:32 smithi069 ceph-mon[35276]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:33.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:33 smithi164 podman[72359]: 2024-05-27 07:14:33.025102747 +0000 UTC m=+0.855191097 container remove 90459befe6b09724c292a73e174ae3619b30cdb4a8b75fbf7f0772e61dfcb27b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T07:14:33.331 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:33 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:14:33.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-mon[35327]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:14:33.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-mon[35327]: mds.? [v2:172.21.15.164:6836/1142577163,v1:172.21.15.164:6837/1142577163] up:boot 2024-05-27T07:14:33.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-mon[35327]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:33.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-mon[35327]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:33.332 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:32 smithi164 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:33.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:32 smithi016 ceph-mon[31415]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:14:33.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:32 smithi016 ceph-mon[31415]: mds.? [v2:172.21.15.164:6836/1142577163,v1:172.21.15.164:6837/1142577163] up:boot 2024-05-27T07:14:33.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:32 smithi016 ceph-mon[31415]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:14:33.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:32 smithi016 ceph-mon[31415]: from='mgr.15297 172.21.15.16:0/450415305' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:14:33.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:32 smithi016 ceph-mon[31415]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:14:33.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:33 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:14:33.831 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:33 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.606s CPU time. 2024-05-27T07:14:34.093 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:33 smithi016 ceph-mon[31415]: pgmap v434: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-27T07:14:34.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:33 smithi164 ceph-mon[35327]: pgmap v434: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-27T07:14:34.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:33 smithi069 ceph-mon[35276]: pgmap v434: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.384+0000 7f9c7de00700 1 -- 172.21.15.16:0/3436223118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 msgr2=0x7f9c781111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.384+0000 7f9c7de00700 1 --2- 172.21.15.16:0/3436223118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c781111c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b0d0 tx=0x7f9c7000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 -- 172.21.15.16:0/3436223118 shutdown_connections 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 --2- 172.21.15.16:0/3436223118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c781111c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 --2- 172.21.15.16:0/3436223118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 0x7f9c781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 --2- 172.21.15.16:0/3436223118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 0x7f9c78109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 -- 172.21.15.16:0/3436223118 >> 172.21.15.16:0/3436223118 conn(0x7f9c78100030 msgr2=0x7f9c78102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 -- 172.21.15.16:0/3436223118 shutdown_connections 2024-05-27T07:14:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.385+0000 7f9c7de00700 1 -- 172.21.15.16:0/3436223118 wait complete. 2024-05-27T07:14:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.386+0000 7f9c7de00700 1 Processor -- start 2024-05-27T07:14:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.386+0000 7f9c7de00700 1 -- start start 2024-05-27T07:14:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 0x7f9c7819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 0x7f9c7819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78114390 con 0x7f9c78109860 2024-05-27T07:14:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78114210 con 0x7f9c781045e0 2024-05-27T07:14:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c7de00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78114510 con 0x7f9c78108f10 2024-05-27T07:14:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c77400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c77400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55894/0 (socket says 172.21.15.16:55894) 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c77400700 1 -- 172.21.15.16:0/1444227915 learned_addr learned my addr 172.21.15.16:0/1444227915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c76a00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 0x7f9c7819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.387+0000 7f9c6e000700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 0x7f9c7819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 -- 172.21.15.16:0/1444227915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 msgr2=0x7f9c7819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 0x7f9c7819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 -- 172.21.15.16:0/1444227915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 msgr2=0x7f9c7819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 0x7f9c7819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 -- 172.21.15.16:0/1444227915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7819fb80 con 0x7f9c78109860 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c77400700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9c70007d10 tx=0x7f9c70007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001f070 con 0x7f9c78109860 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c70003e10 con 0x7f9c78109860 2024-05-27T07:14:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.388+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001b820 con 0x7f9c78109860 2024-05-27T07:14:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.389+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7819fe10 con 0x7f9c78109860 2024-05-27T07:14:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.389+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781a02a0 con 0x7f9c78109860 2024-05-27T07:14:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.390+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c780512b0 con 0x7f9c78109860 2024-05-27T07:14:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.393+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9c70022070 con 0x7f9c78109860 2024-05-27T07:14:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.394+0000 7f9c7ca00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 0x7f9c5c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:14:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.394+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9437+0+0 (secure 0 0 0) 0x7f9c7000f020 con 0x7f9c78109860 2024-05-27T07:14:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.395+0000 7f9c76a00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 0x7f9c5c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:14:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.395+0000 7f9c76a00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 0x7f9c5c0b4f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c68005950 tx=0x7f9c680058e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:14:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.397+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c70029020 con 0x7f9c78109860 2024-05-27T07:14:34.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.657+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c7802e5e0 con 0x7f9c78109860 2024-05-27T07:14:34.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.658+0000 7f9c7ca00700 1 -- 172.21.15.16:0/1444227915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v113) ==== 81+0+944 (secure 0 0 0) 0x7f9c700ab9c0 con 0x7f9c78109860 2024-05-27T07:14:34.648 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/702672487 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2755801148 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3562581452 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1801942445 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2589421123 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/454643589 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3708593763 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3327639895 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3317326035 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3906409731 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4243270602 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2755801148 2024-05-28T07:00:53.625719+0000 2024-05-27T07:14:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/4120962675 2024-05-28T06:44:00.893187+0000 2024-05-27T07:14:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2390178341 2024-05-28T06:44:37.876629+0000 2024-05-27T07:14:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 msgr2=0x7f9c5c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 0x7f9c5c0b4f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c68005950 tx=0x7f9c680058e0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 msgr2=0x7f9c7819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:14:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9c70007d10 tx=0x7f9c70007df0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 shutdown_connections 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:6800/1617637164,v1:172.21.15.16:6801/1617637164] conn(0x7f9c5c0b2a70 0x7f9c5c0b4f80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78109860 0x7f9c7819f590 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c781045e0 0x7f9c7819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 --2- 172.21.15.16:0/1444227915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c78108f10 0x7f9c7819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 >> 172.21.15.16:0/1444227915 conn(0x7f9c78100030 msgr2=0x7f9c78108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.662+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 shutdown_connections 2024-05-27T07:14:34.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-27T07:14:34.663+0000 7f9c7de00700 1 -- 172.21.15.16:0/1444227915 wait complete. 2024-05-27T07:14:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:listed 16 entries 2024-05-27T07:14:35.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:34 smithi164 ceph-mon[35327]: from='client.? 172.21.15.16:0/1444227915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:35.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:34 smithi016 ceph-mon[31415]: from='client.? 172.21.15.16:0/1444227915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:35.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:34 smithi069 ceph-mon[35276]: from='client.? 172.21.15.16:0/1444227915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:14:35.639 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-05-27T07:14:35.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:14:35.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-27T07:14:35.748 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T07:14:35.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[89660]: fuse finished with error 0 and tester_r 0 2024-05-27T07:14:36.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:35 smithi164 ceph-mon[35327]: pgmap v435: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:14:36.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:35 smithi016 ceph-mon[31415]: pgmap v435: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:14:36.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:35 smithi069 ceph-mon[35276]: pgmap v435: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-27T07:14:38.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:37 smithi164 ceph-mon[35327]: pgmap v436: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:38.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:37 smithi016 ceph-mon[31415]: pgmap v436: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:38.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:37 smithi069 ceph-mon[35276]: pgmap v436: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 102 B/s wr, 8 op/s 2024-05-27T07:14:40.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:40 smithi164 ceph-mon[35327]: pgmap v437: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 307 B/s wr, 9 op/s 2024-05-27T07:14:40.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:40 smithi016 ceph-mon[31415]: pgmap v437: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 307 B/s wr, 9 op/s 2024-05-27T07:14:40.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:40 smithi069 ceph-mon[35276]: pgmap v437: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 307 B/s wr, 9 op/s 2024-05-27T07:14:41.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:41 smithi164 ceph-mon[35327]: pgmap v438: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 409 B/s wr, 9 op/s 2024-05-27T07:14:41.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:41 smithi016 ceph-mon[31415]: pgmap v438: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 409 B/s wr, 9 op/s 2024-05-27T07:14:41.367 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:41 smithi069 ceph-mon[35276]: pgmap v438: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 409 B/s wr, 9 op/s 2024-05-27T07:14:41.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:14:41.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:14:41.850 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T07:14:42.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:14:42.090 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-27T07:14:42.300 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T07:14:42.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:14:42.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:14:42.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:14:42.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-05-27T07:14:42.509 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T07:14:42.522 INFO:tasks.cephadm:Teardown begin 2024-05-27T07:14:42.523 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump' 2024-05-27T07:14:42.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:14:42.600 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:14:42.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:14:42.793 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T07:14:42.793 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T07:14:42.872 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T07:14:42.929 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T07:14:42.987 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T07:14:42.988 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-27T07:14:42.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a 2024-05-27T07:14:43.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:43 smithi016 systemd[1]: Stopping Ceph mon.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:43.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:43 smithi016 ceph-mon[31415]: pgmap v439: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-27T07:14:43.544 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:43 smithi069 ceph-mon[35276]: pgmap v439: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-27T07:14:43.580 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:43 smithi164 ceph-mon[35327]: pgmap v439: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-27T07:14:43.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a[31411]: 2024-05-27T07:14:43.588+0000 7f29f8c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:14:43.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a[31411]: 2024-05-27T07:14:43.588+0000 7f29f8c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T07:14:44.081 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-05-27T07:14:44.081 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: Stopped Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:44.081 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.606s CPU time. 2024-05-27T07:14:44.082 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Start request repeated too quickly. 2024-05-27T07:14:44.082 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:14:44.082 INFO:journalctl@ceph.mds.c.smithi164.stdout:May 27 07:14:43 smithi164 systemd[1]: Failed to start Ceph mds.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:14:44.292 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:43 smithi016 podman[113634]: 2024-05-27 07:14:43.95060367 +0000 UTC m=+0.514754761 container died 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T07:14:44.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:44 smithi016 podman[113634]: 2024-05-27 07:14:44.547569775 +0000 UTC m=+1.111720859 container cleanup 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:14:44.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:44 smithi016 bash[113634]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a 2024-05-27T07:14:45.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 27 07:14:44 smithi016 podman[113648]: 2024-05-27 07:14:44.850196185 +0000 UTC m=+0.884782284 container remove 2f7380fe993309e508b5952f5433b0f56084da8cc1d05357c44047a8e3155692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:14:45.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.a.service' 2024-05-27T07:14:45.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:45.450 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-27T07:14:45.450 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-27T07:14:45.450 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.b 2024-05-27T07:14:45.867 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:45 smithi069 systemd[1]: Stopping Ceph mon.b for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:46.388 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b[35272]: 2024-05-27T07:14:46.139+0000 7f78df600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:14:46.388 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b[35272]: 2024-05-27T07:14:46.139+0000 7f78df600700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-27T07:14:46.704 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:46 smithi069 podman[72150]: 2024-05-27 07:14:46.397811978 +0000 UTC m=+0.441256810 container died d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T07:14:47.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:46 smithi069 podman[72150]: 2024-05-27 07:14:46.858606476 +0000 UTC m=+0.902051298 container cleanup d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T07:14:47.118 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:46 smithi069 bash[72150]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b 2024-05-27T07:14:47.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 27 07:14:47 smithi069 podman[72162]: 2024-05-27 07:14:47.318680866 +0000 UTC m=+0.897279914 container remove d4af107a3aec701b5ba7fdde5485eecdad0821a68869672c7848e08c820e482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T07:14:47.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.b.service' 2024-05-27T07:14:48.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:48.015 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-27T07:14:48.015 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-27T07:14:48.016 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.c 2024-05-27T07:14:48.552 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:48 smithi164 systemd[1]: Stopping Ceph mon.c for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:48.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c[35323]: 2024-05-27T07:14:48.558+0000 7f1284000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:14:48.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c[35323]: 2024-05-27T07:14:48.558+0000 7f1284000700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T07:14:48.831 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:48 smithi164 podman[72483]: 2024-05-27 07:14:48.806994587 +0000 UTC m=+0.372912126 container died 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T07:14:49.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:49 smithi164 podman[72483]: 2024-05-27 07:14:49.27078948 +0000 UTC m=+0.836707042 container cleanup 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T07:14:49.581 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:49 smithi164 bash[72483]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c 2024-05-27T07:14:50.040 INFO:journalctl@ceph.mon.c.smithi164.stdout:May 27 07:14:49 smithi164 podman[72497]: 2024-05-27 07:14:49.61397585 +0000 UTC m=+0.789846257 container remove 0fc103f84c79689e978bc752902540d2dc09d8158679fdd0948d8b9a814f8f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T07:14:50.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mon.c.service' 2024-05-27T07:14:50.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:50.199 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-27T07:14:50.199 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-27T07:14:50.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T07:14:50.609 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:50 smithi016 systemd[1]: Stopping Ceph mgr.x for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:51.359 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x[31644]: 2024-05-27T07:14:50.894+0000 7f2aec600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:14:51.359 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:50 smithi016 podman[113755]: 2024-05-27 07:14:50.998983192 +0000 UTC m=+0.310255103 container died 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T07:14:51.965 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:51 smithi016 podman[113755]: 2024-05-27 07:14:51.679731806 +0000 UTC m=+0.991003707 container cleanup 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T07:14:51.965 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:51 smithi016 bash[113755]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x 2024-05-27T07:14:52.358 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 27 07:14:51 smithi016 podman[113768]: 2024-05-27 07:14:51.973698332 +0000 UTC m=+0.950155595 container remove 91ab9bd87972945e1a34e21afb8e74bcc7273fc0540dfc5c14384c7a50775c7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T07:14:52.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.x.service' 2024-05-27T07:14:52.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:52.637 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-27T07:14:52.638 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-27T07:14:52.638 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.y 2024-05-27T07:14:53.118 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:52 smithi069 systemd[1]: Stopping Ceph mgr.y for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:53.611 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:53 smithi069 podman[72282]: 2024-05-27 07:14:53.280749765 +0000 UTC m=+0.227389371 container died c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-27T07:14:53.867 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:53 smithi069 podman[72282]: 2024-05-27 07:14:53.798082469 +0000 UTC m=+0.744722077 container cleanup c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:14:53.868 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:53 smithi069 bash[72282]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y 2024-05-27T07:14:54.368 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:53 smithi069 podman[72297]: 2024-05-27 07:14:53.982812376 +0000 UTC m=+0.686366644 container remove c263c450001cfea19bcb66913d5235c1d7ee0f34ea8e6320a5dd5a726cf756ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:14:54.368 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 27 07:14:53 smithi069 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-27T07:14:54.425 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.y.service' 2024-05-27T07:14:54.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:54.529 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-27T07:14:54.529 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-27T07:14:54.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.z 2024-05-27T07:14:55.081 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:54 smithi164 systemd[1]: Stopping Ceph mgr.z for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:55.517 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:55 smithi164 podman[72617]: 2024-05-27 07:14:55.226962719 +0000 UTC m=+0.228903047 container died 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T07:14:55.831 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:55 smithi164 podman[72617]: 2024-05-27 07:14:55.712175085 +0000 UTC m=+0.714115380 container cleanup 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:14:55.831 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:55 smithi164 bash[72617]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z 2024-05-27T07:14:56.331 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:55 smithi164 podman[72631]: 2024-05-27 07:14:55.905405278 +0000 UTC m=+0.661103621 container remove 1f7404f5138d33c7032f588f79c9e392494027cc40977c6a2d1104445501a415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:14:56.331 INFO:journalctl@ceph.mgr.z.smithi164.stdout:May 27 07:14:55 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-27T07:14:56.402 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mgr.z.service' 2024-05-27T07:14:56.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:14:56.514 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-27T07:14:56.514 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T07:14:56.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.0 2024-05-27T07:14:56.858 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:14:56 smithi016 systemd[1]: Stopping Ceph osd.0 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:14:57.608 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:14:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0[43598]: 2024-05-27T07:14:57.164+0000 7fc840e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:14:57.609 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:14:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0[43598]: 2024-05-27T07:14:57.164+0000 7fc840e00700 -1 osd.0 113 *** Got signal Terminated *** 2024-05-27T07:14:57.609 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:14:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0[43598]: 2024-05-27T07:14:57.164+0000 7fc840e00700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:03.359 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:03 smithi016 podman[113873]: 2024-05-27 07:15:03.107597577 +0000 UTC m=+6.098805392 container died f51711abef3f36632d6608e55c2c5a6766dd4fee54ff28f5b599a6b57ebbbd67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:15:04.062 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:03 smithi016 podman[113873]: 2024-05-27 07:15:03.786185346 +0000 UTC m=+6.777393225 container cleanup f51711abef3f36632d6608e55c2c5a6766dd4fee54ff28f5b599a6b57ebbbd67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T07:15:04.063 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:03 smithi016 bash[113873]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0 2024-05-27T07:15:04.358 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:04 smithi016 podman[113891]: 2024-05-27 07:15:04.071675862 +0000 UTC m=+0.929732396 container remove f51711abef3f36632d6608e55c2c5a6766dd4fee54ff28f5b599a6b57ebbbd67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:05.117 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:04 smithi016 podman[113989]: 2024-05-27 07:15:04.78205422 +0000 UTC m=+0.049811299 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:05.118 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:04 smithi016 podman[113989]: 2024-05-27 07:15:04.914542354 +0000 UTC m=+0.182299448 container create 2f4767d13e47aa32fe204d3e6dd497c8de15cfa0df989508bdb62d537a1c02d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T07:15:05.433 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:05 smithi016 podman[113989]: 2024-05-27 07:15:05.260749655 +0000 UTC m=+0.528506832 container init 2f4767d13e47aa32fe204d3e6dd497c8de15cfa0df989508bdb62d537a1c02d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD) 2024-05-27T07:15:05.433 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:05 smithi016 podman[113989]: 2024-05-27 07:15:05.273366957 +0000 UTC m=+0.541124101 container start 2f4767d13e47aa32fe204d3e6dd497c8de15cfa0df989508bdb62d537a1c02d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:15:05.858 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:05 smithi016 podman[113989]: 2024-05-27 07:15:05.442942741 +0000 UTC m=+0.710699854 container attach 2f4767d13e47aa32fe204d3e6dd497c8de15cfa0df989508bdb62d537a1c02d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:15:08.324 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 27 07:15:07 smithi016 podman[113989]: 2024-05-27 07:15:07.906129542 +0000 UTC m=+3.173886686 container died 2f4767d13e47aa32fe204d3e6dd497c8de15cfa0df989508bdb62d537a1c02d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:08.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T07:15:09.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:09.046 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T07:15:09.046 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T07:15:09.046 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.1 2024-05-27T07:15:09.602 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:09 smithi069 systemd[1]: Stopping Ceph osd.1 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:09.868 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1[39824]: 2024-05-27T07:15:09.608+0000 7f1d3ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:09.868 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1[39824]: 2024-05-27T07:15:09.608+0000 7f1d3ea00700 -1 osd.1 113 *** Got signal Terminated *** 2024-05-27T07:15:09.868 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1[39824]: 2024-05-27T07:15:09.608+0000 7f1d3ea00700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:15.725 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:15 smithi069 podman[72420]: 2024-05-27 07:15:15.416244586 +0000 UTC m=+5.956237013 container stop 91d6a8d5594de226a37b3ba13400c50fabff0e12ba643e15f2d2aeb8b3d92e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T07:15:15.725 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:15 smithi069 podman[72420]: 2024-05-27 07:15:15.417240044 +0000 UTC m=+5.957232472 container died 91d6a8d5594de226a37b3ba13400c50fabff0e12ba643e15f2d2aeb8b3d92e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T07:15:16.118 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:15 smithi069 podman[72420]: 2024-05-27 07:15:15.861809102 +0000 UTC m=+6.401801567 container cleanup 91d6a8d5594de226a37b3ba13400c50fabff0e12ba643e15f2d2aeb8b3d92e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:16.118 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:15 smithi069 bash[72420]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1 2024-05-27T07:15:16.617 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:16 smithi069 podman[72437]: 2024-05-27 07:15:16.166866469 +0000 UTC m=+0.715384879 container remove 91d6a8d5594de226a37b3ba13400c50fabff0e12ba643e15f2d2aeb8b3d92e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:15:17.227 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:16 smithi069 podman[72524]: 2024-05-27 07:15:16.819421362 +0000 UTC m=+0.036035232 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:17.228 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:16 smithi069 podman[72524]: 2024-05-27 07:15:16.955875256 +0000 UTC m=+0.172489146 container create 33d2a35f28b81cdbb004bcf77e46adb63e161e75ada8cba0dc6a6efab642b9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T07:15:17.228 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:17 smithi069 podman[72524]: 2024-05-27 07:15:17.236896208 +0000 UTC m=+0.453510105 container init 33d2a35f28b81cdbb004bcf77e46adb63e161e75ada8cba0dc6a6efab642b9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-27T07:15:17.617 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:17 smithi069 podman[72524]: 2024-05-27 07:15:17.249365306 +0000 UTC m=+0.465979240 container start 33d2a35f28b81cdbb004bcf77e46adb63e161e75ada8cba0dc6a6efab642b9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T07:15:17.618 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:17 smithi069 podman[72524]: 2024-05-27 07:15:17.324957981 +0000 UTC m=+0.541571868 container attach 33d2a35f28b81cdbb004bcf77e46adb63e161e75ada8cba0dc6a6efab642b9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T07:15:18.118 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 27 07:15:17 smithi069 podman[72524]: 2024-05-27 07:15:17.861219241 +0000 UTC m=+1.077833171 container died 33d2a35f28b81cdbb004bcf77e46adb63e161e75ada8cba0dc6a6efab642b9ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:15:18.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T07:15:18.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:18.720 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T07:15:18.720 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T07:15:18.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.2 2024-05-27T07:15:19.080 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:18 smithi164 systemd[1]: Stopping Ceph osd.2 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:19.582 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2[40411]: 2024-05-27T07:15:19.305+0000 7f4a31200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:19.582 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2[40411]: 2024-05-27T07:15:19.305+0000 7f4a31200700 -1 osd.2 113 *** Got signal Terminated *** 2024-05-27T07:15:19.582 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2[40411]: 2024-05-27T07:15:19.305+0000 7f4a31200700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:23.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:24.332 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:23.931+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:24.458 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:24.168+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:24.740 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:24.467+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:24.911 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:24.592+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:24.912 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:24.919+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:24.912 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:24.779+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:25.109 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:24.748+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:25.331 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:25 smithi164 podman[72739]: 2024-05-27 07:15:25.129423697 +0000 UTC m=+5.948199465 container died 36a8b71970a33d6a7272eefada53facea7cda604e19913afb99b62f4460cc122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:15:25.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:25.454+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:25.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:25.210+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:25.735 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:25.617+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:25.735 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:25 smithi164 podman[72739]: 2024-05-27 07:15:25.576890909 +0000 UTC m=+6.395666716 container cleanup 36a8b71970a33d6a7272eefada53facea7cda604e19913afb99b62f4460cc122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T07:15:25.736 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:25 smithi164 bash[72739]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2 2024-05-27T07:15:26.080 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:25 smithi164 podman[72755]: 2024-05-27 07:15:25.803535807 +0000 UTC m=+0.642856993 container remove 36a8b71970a33d6a7272eefada53facea7cda604e19913afb99b62f4460cc122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T07:15:26.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:25.743+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:26.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:25.896+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:26.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:25.699+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:26.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:26.436+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:26.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:26.194+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:26.849 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:26 smithi164 podman[72840]: 2024-05-27 07:15:26.508000894 +0000 UTC m=+0.032709643 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:26.849 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:26 smithi164 podman[72840]: 2024-05-27 07:15:26.654602439 +0000 UTC m=+0.179311221 container create f13e4eb762191cddb1e5fb7b810305b19f086f9d8588e06af7779f024d7a7667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:26.849 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:26.772+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:26.850 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:26.644+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:27.107 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:26 smithi164 podman[72840]: 2024-05-27 07:15:26.997863148 +0000 UTC m=+0.522571960 container init f13e4eb762191cddb1e5fb7b810305b19f086f9d8588e06af7779f024d7a7667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:15:27.107 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:27 smithi164 podman[72840]: 2024-05-27 07:15:27.009497288 +0000 UTC m=+0.534206090 container start f13e4eb762191cddb1e5fb7b810305b19f086f9d8588e06af7779f024d7a7667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:27.107 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:26.857+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:27.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:26.729+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:27.464 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:27.173+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:27.858 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:27.719+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:27.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:27.474+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:28.040 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:27.610+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:28.041 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:27.803+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:28.041 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:27 smithi164 podman[72840]: 2024-05-27 07:15:27.116192864 +0000 UTC m=+0.640901712 container attach f13e4eb762191cddb1e5fb7b810305b19f086f9d8588e06af7779f024d7a7667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T07:15:28.041 INFO:journalctl@ceph.osd.2.smithi164.stdout:May 27 07:15:27 smithi164 podman[72840]: 2024-05-27 07:15:27.63609835 +0000 UTC m=+1.160807145 container died f13e4eb762191cddb1e5fb7b810305b19f086f9d8588e06af7779f024d7a7667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:15:28.041 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:27.847+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:28.431 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:28.145+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:28.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T07:15:28.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:28.580 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T07:15:28.580 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T07:15:28.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.3 2024-05-27T07:15:28.707 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:28.441+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:28.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:28.647+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:28.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:28.611+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:29.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:28 smithi016 systemd[1]: Stopping Ceph osd.3 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:29.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:28.728+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:29.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:28.841+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:29.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:28.868+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:29.480 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:29.194+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:29.480 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:29.249+0000 7f7dda600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:29.480 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:29.249+0000 7f7dda600700 -1 osd.3 113 *** Got signal Terminated *** 2024-05-27T07:15:29.480 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:29.249+0000 7f7dda600700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:29.765 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:29.489+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:29.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:29.570+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:30.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:29.846+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:30.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:29.684+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:30.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:29.873+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:30.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:29.773+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:30.459 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:30.211+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:30.807 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:30.468+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:30.816 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:30.571+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:31.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:30.871+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:31.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:30.734+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:31.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:30.922+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:31.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:30.817+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:31.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:31.448+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:31.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:31.213+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:31.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:31.537+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:31.982 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:31.734+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:31.983 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:31.845+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:31.983 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:31.894+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:32.107 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:31.856+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:32.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:32.424+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:32.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:32.186+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:32.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:32.585+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:33.021 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:32.840+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:33.021 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:32.697+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:33.021 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:32.876+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:33.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:32.858+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:33.461 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:33.187+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:33.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:33.469+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:33.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:33.594+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:34.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:33.859+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:34.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:33.671+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:34.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:33.849+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:34.142 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3[50532]: 2024-05-27T07:15:33.879+0000 7f7dd0e00700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.724018+0000 front 2024-05-27T07:15:01.723909+0000 (oldest deadline 2024-05-27T07:15:24.023944+0000) 2024-05-27T07:15:34.485 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:34 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:34.179+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:34.828 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:34.659+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:34.847 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:34 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:34.493+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:35.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:34.836+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:35.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:34.891+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:35.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:35 smithi016 podman[114093]: 2024-05-27 07:15:35.077574455 +0000 UTC m=+5.965489559 container died 4c8228784b228c97c889a7a5a6f68d8a8afe16561d5b84b21cc14d4901d397ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T07:15:35.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:34.631+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:35.447 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:35 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:35.158+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:35.701 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:35 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:35.462+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:35.995 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:35 smithi016 podman[114093]: 2024-05-27 07:15:35.71076003 +0000 UTC m=+6.598675124 container cleanup 4c8228784b228c97c889a7a5a6f68d8a8afe16561d5b84b21cc14d4901d397ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T07:15:35.995 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:35 smithi016 bash[114093]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3 2024-05-27T07:15:36.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:35.809+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:36.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:35.675+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:36.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:35.909+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:36.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:35.661+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:36.358 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:36 smithi016 podman[114119]: 2024-05-27 07:15:36.004482707 +0000 UTC m=+0.900396391 container remove 4c8228784b228c97c889a7a5a6f68d8a8afe16561d5b84b21cc14d4901d397ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:36.358 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:36.146+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:36.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:36 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:36.470+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:37.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:36.766+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:37.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:36.933+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:37.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:36.714+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:37.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:36.670+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:37.177 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:36 smithi016 podman[114210]: 2024-05-27 07:15:36.785360184 +0000 UTC m=+0.034010119 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:37.177 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:36 smithi016 podman[114210]: 2024-05-27 07:15:36.922200856 +0000 UTC m=+0.170850819 container create 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T07:15:37.456 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:37.186+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:37.457 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:37 smithi016 podman[114210]: 2024-05-27 07:15:37.259441979 +0000 UTC m=+0.508091991 container init 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T07:15:37.457 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:37 smithi016 podman[114210]: 2024-05-27 07:15:37.271187387 +0000 UTC m=+0.519837326 container start 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:15:37.457 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:37 smithi016 podman[114210]: 2024-05-27 07:15:37.366754704 +0000 UTC m=+0.615404696 container attach 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:15:37.732 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:37 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:37.465+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:37.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:37.648+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:37.868 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:37.409+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:38.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:37.890+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:38.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:37.689+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:38.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:37.689+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:38.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:37.781+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:38.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:37.781+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:38.108 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:37 smithi016 podman[114236]: 2024-05-27 07:15:37.794412328 +0000 UTC m=+0.034912225 container died 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:15:38.483 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:38 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:38.226+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:38.743 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 27 07:15:38 smithi016 podman[114236]: 2024-05-27 07:15:38.721594318 +0000 UTC m=+0.962094252 container remove 403d2a4127ce6c87d8e7a202e8e2fa3f54e7c2b875db6ebafd10a6a4f869fa02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T07:15:38.743 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:38 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:38.492+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:38.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T07:15:38.867 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:38.413+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:38.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:38.613+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:38.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:38.876 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T07:15:38.876 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T07:15:38.876 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.4 2024-05-27T07:15:39.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:38.722+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:39.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:38.722+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:39.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:38.787+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:39.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:38.787+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:39.082 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:38.904+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:39.082 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:38.904+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:39.367 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:39 smithi069 systemd[1]: Stopping Ceph osd.4 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:39.525 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:39.247+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:39.694 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:39.652+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:39.695 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:39.432+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:39.695 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:39.472+0000 7f202f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:39.695 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:39.472+0000 7f202f600700 -1 osd.4 113 *** Got signal Terminated *** 2024-05-27T07:15:39.695 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:39.472+0000 7f202f600700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:39.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:39.535+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:39.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:39 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:39.535+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:39.739+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:39.739+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:39.683+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:39.683+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:39.919+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:40.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:39.919+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:40.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:39.703+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:40.528 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:40.226+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:40.528 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:40.226+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:40.735 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:40.443+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:40.736 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:40.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:40.736 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:40.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:40.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:40.537+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:40.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:40 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:40.537+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:40.904 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:40.737+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:40.904 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:40.737+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:40.904 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:40.655+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:40.904 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:40.655+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:41.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:40.744+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:41.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:40.912+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:41.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:40.912+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:41.534 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:41.264+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:41.534 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:41.264+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:41.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:41.543+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:41.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:41 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:41.544+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:41.867 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:41.470+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:41.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:41.704+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:41.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:41.618+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:41.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:41.618+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:41.927 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:41.773+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:41.928 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:41.773+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:41.928 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:41.662+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:41.928 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:41.662+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:42.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:41.936+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:42.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:41.936+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:42.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:42.496+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:42.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:42.496+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:42.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:42.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:42.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:42 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:42.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:42.867 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:42.475+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:42.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:42.713+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:42.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:42.592+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:42.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:42.593+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:42.954 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:42.763+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:42.954 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:42.763+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:42.954 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:42.652+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:42.954 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:42.652+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:43.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:42.962+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:43.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:42.962+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:43.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:43.537+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:43.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:43.537+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:43.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:43.351+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:43.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:43 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:43.351+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:43.722 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:43.463+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:43.723 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:43.574+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:43.723 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:43.574+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:43.929 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:43.769+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:43.929 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:43.769+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:43.930 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:43.658+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:43.930 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:43.658+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:44.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:43 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:43.731+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:44.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:43.938+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:44.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:43.938+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:44.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:44.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:44.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:44.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:44.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:44.335+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:44.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:44 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:44.335+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:44.711 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4[46394]: 2024-05-27T07:15:44.427+0000 7f2025e00700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.199013+0000 front 2024-05-27T07:15:13.199400+0000 (oldest deadline 2024-05-27T07:15:36.699104+0000) 2024-05-27T07:15:44.712 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:44.560+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:44.712 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:44.560+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:44.957 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:44.656+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:44.957 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:44.656+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:44.958 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:44.765+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:44.958 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:44.765+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:45.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:44 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:44.720+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:45.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:44.965+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:45.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:44.965+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:45.565 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:45.292+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:45.565 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:45.292+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:45.720 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:45 smithi069 podman[72642]: 2024-05-27 07:15:45.428194561 +0000 UTC m=+6.103204334 container died de40c2f6638350b31fa603ca06d6e3a0a6b87389361e194eb04ea63f4b881822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T07:15:45.721 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:45 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:45.677+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:45.721 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:45 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:45.546+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:45.721 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:45 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:45.546+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:45.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:45.574+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:45.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:45 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:45.574+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:45.976 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:45.653+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:45.977 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:45.653+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:45.977 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:45.780+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:45.977 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:45.780+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:46.118 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:45 smithi069 podman[72642]: 2024-05-27 07:15:45.857934366 +0000 UTC m=+6.532944119 container cleanup de40c2f6638350b31fa603ca06d6e3a0a6b87389361e194eb04ea63f4b881822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T07:15:46.118 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:45 smithi069 bash[72642]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4 2024-05-27T07:15:46.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:45.985+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:46.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:45.985+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:46.534 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:46 smithi069 podman[72662]: 2024-05-27 07:15:46.193049235 +0000 UTC m=+0.731427989 container remove de40c2f6638350b31fa603ca06d6e3a0a6b87389361e194eb04ea63f4b881822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:15:46.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:46.557+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:46.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:46.557+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:46.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:46.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:46.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:46.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:46.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:46 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:46.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:46.842 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:46 smithi069 podman[72749]: 2024-05-27 07:15:46.697674501 +0000 UTC m=+0.032946955 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:46.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:46.648+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:46.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:46.648+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:46.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:46.544+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:46.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:46.544+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:46.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:46 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:46.544+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:47.024 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:46.668+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:47.024 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:46.668+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:47.024 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:46.787+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:47.024 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:46.787+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:47.117 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:46 smithi069 podman[72749]: 2024-05-27 07:15:46.852380268 +0000 UTC m=+0.187652756 container create 94967deb00d3cdae7cbfb2c532b9ee820ec015202a2c02532da98afec10f9966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T07:15:47.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:47.032+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:47.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:47.032+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:47.560 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:47 smithi069 podman[72749]: 2024-05-27 07:15:47.133183622 +0000 UTC m=+0.468456120 container init 94967deb00d3cdae7cbfb2c532b9ee820ec015202a2c02532da98afec10f9966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T07:15:47.561 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:47 smithi069 podman[72749]: 2024-05-27 07:15:47.14563665 +0000 UTC m=+0.480909106 container start 94967deb00d3cdae7cbfb2c532b9ee820ec015202a2c02532da98afec10f9966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T07:15:47.561 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:47 smithi069 podman[72749]: 2024-05-27 07:15:47.22157349 +0000 UTC m=+0.556846001 container attach 94967deb00d3cdae7cbfb2c532b9ee820ec015202a2c02532da98afec10f9966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T07:15:47.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:47.549+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:47.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:47.549+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:47.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:47.371+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:47.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:47.371+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:47.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:47 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:47.371+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:47.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:47.667+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:47.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:47.667+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:47.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:47.569+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:47.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:47.569+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:47.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:47 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:47.569+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:47.868 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 27 07:15:47 smithi069 podman[72749]: 2024-05-27 07:15:47.615980183 +0000 UTC m=+0.951252677 container died 94967deb00d3cdae7cbfb2c532b9ee820ec015202a2c02532da98afec10f9966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:15:48.029 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:47.693+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:48.029 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:47.693+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:48.029 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:47.782+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:48.029 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:47.782+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:48.029 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:47.782+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:48.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:48.037+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:48.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:48.037+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:48.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T07:15:48.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:48.511 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T07:15:48.511 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T07:15:48.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.5 2024-05-27T07:15:48.618 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:48.610+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:48.618 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:48.610+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:48.618 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:48.610+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:48.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:48.529+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:48.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:48.529+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:48.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:48.387+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:48.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:48.387+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:48.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:48 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:48.387+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:49.077 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:48.681+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:49.077 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:48.681+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:49.077 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:48 smithi164 systemd[1]: Stopping Ceph osd.5 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:49.078 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:48.797+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:49.078 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:48.797+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:49.078 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:48.797+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:49.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:48.646+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:49.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:48 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:48.646+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:49.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.139+0000 7f20cd000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:49.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.139+0000 7f20cd000700 -1 osd.5 113 *** Got signal Terminated *** 2024-05-27T07:15:49.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.139+0000 7f20cd000700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:49.332 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:49.085+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:49.332 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:49.085+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:49.332 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:49.085+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:49.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:49.399+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:49.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:49.399+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:49.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:49.399+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:49.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:49.556+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:49.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:49.556+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:49.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:49 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:49.556+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:49.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:49.609+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:49.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:49.609+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:49.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:49.623+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:49.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:49.623+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:49.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:49 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:49.623+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:50.039 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:49.649+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:50.039 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:49.649+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:50.040 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.843+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:50.040 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.843+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:50.040 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:49.843+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:50.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:50.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:50.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:50.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:50.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:50.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:50.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:50.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:50.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:50.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:50.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:50.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:50.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:50.583+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:50.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:50.583+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:50.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:50 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:50.583+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:50.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:50.560+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:50.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:50.560+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:50.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:50.668+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:50.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:50.668+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:50.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:50 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:50.668+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:51.069 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:50.660+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:51.069 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:50.660+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:51.070 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:50.660+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:51.070 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:50.849+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:51.070 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:50.849+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:51.070 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:50.849+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:51.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:51.078+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:51.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:51.078+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:51.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:51.078+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:51.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:51.408+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:51.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:51.408+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:51.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:51.408+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:51.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:51.547+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:51.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:51.547+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:51.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:51 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:51.547+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:51.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:51.605+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:51.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:51.605+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:51.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:51.694+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:51.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:51.694+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:51.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:51 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:51.694+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:52.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:51.646+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:52.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:51.646+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:52.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:51.646+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:52.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:51.834+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:52.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:51.834+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:52.082 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:51.834+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:52.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:52.117+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:52.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:52.117+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:52.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:52.117+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:52.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:52.362+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:52.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:52.362+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:52.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:52.362+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:52.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:52.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:52.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:52.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:52.609 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:52 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:52.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:53.075 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:52.792+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:53.075 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:52.792+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:53.075 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:52.792+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:53.076 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:52.678+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:53.076 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:52.678+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:53.076 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:52.678+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:53.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:52.641+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:53.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:52.641+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:53.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:52.696+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:53.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:52.696+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:53.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:52 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:52.696+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:53.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:53.084+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:53.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:53.084+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:53.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:53.084+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:53.587 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:53.346+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:53.587 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:53.346+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:53.587 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:53.346+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:53.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:53.596+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:53.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:53.596+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:53.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:53 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:53.596+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:53.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:53 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:53.615+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:53.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:53 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:53.615+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:53.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:53 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:53.652+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:53.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:53 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:53.652+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:53.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:53 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:53.652+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:54.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:53.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:54.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:53.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:54.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:53.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:54.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:53.789+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.847474+0000 front 2024-05-27T07:14:58.847718+0000 (oldest deadline 2024-05-27T07:15:24.147375+0000) 2024-05-27T07:15:54.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:53.789+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.353204+0000 front 2024-05-27T07:15:13.353351+0000 (oldest deadline 2024-05-27T07:15:37.453316+0000) 2024-05-27T07:15:54.082 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:53 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5[47026]: 2024-05-27T07:15:53.789+0000 7f20c8800700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:20.955362+0000 front 2024-05-27T07:15:20.955591+0000 (oldest deadline 2024-05-27T07:15:47.356062+0000) 2024-05-27T07:15:54.554 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:54.562+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:54.554 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:54.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:54.554 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:54.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:54.554 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:54.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:54.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:54.105+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:54.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:54.105+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:54.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:54.105+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:54.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:54.562+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:54.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:54 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:54.562+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:55.054 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:54.638+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:55.054 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:54.638+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:55.055 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:54 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:54.638+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:55.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:54 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:54.664+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:55.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:54 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:54.664+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:55.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:54 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:54.665+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:55.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:54 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:54.665+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:55.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:54 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:54.665+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:55.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:55 smithi164 podman[72959]: 2024-05-27 07:15:55.193687965 +0000 UTC m=+6.204119710 container died 55ef163c7e9caa7d7f387534dcfdb0dbf1a9776e5f9f60ae97584a8ded041e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T07:15:55.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:55.063+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:55.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:55.063+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:55.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:55.063+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:55.563 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:55.285+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:55.563 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:55.285+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:55.563 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:55.285+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:55.830 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:55.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:55.848 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:55.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:55.848 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:55 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:55.666+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:55.848 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:55 smithi164 podman[72959]: 2024-05-27 07:15:55.682241258 +0000 UTC m=+6.692673000 container cleanup 55ef163c7e9caa7d7f387534dcfdb0dbf1a9776e5f9f60ae97584a8ded041e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T07:15:55.848 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:55 smithi164 bash[72959]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5 2024-05-27T07:15:55.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:55.571+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:55.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:55.571+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:55.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:55 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:55.571+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:55.703+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:55.703+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:55.691+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:55.691+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:55.691+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:56.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:55 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:55.691+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:15:56.330 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:55 smithi164 podman[72977]: 2024-05-27 07:15:55.950521002 +0000 UTC m=+0.743653608 container remove 55ef163c7e9caa7d7f387534dcfdb0dbf1a9776e5f9f60ae97584a8ded041e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T07:15:56.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:56.030+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:56.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:56.030+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:56.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:56.030+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:56.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:56.030+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:15:56.597 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:56.279+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:56.597 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:56.279+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:56.597 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:56.279+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:56.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:56.605+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:56.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:56.605+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:56.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:56 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:56.605+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:56.923 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:56 smithi164 podman[73069]: 2024-05-27 07:15:56.649109302 +0000 UTC m=+0.032977327 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:15:56.923 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:56 smithi164 podman[73069]: 2024-05-27 07:15:56.768275864 +0000 UTC m=+0.152143930 container create 2bff1c5b5551bd20e57651faafb148fd122b9b3c935a69af8e682067677d90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:15:56.924 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:56.677+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:56.924 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:56.677+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:56.924 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:56 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:56.677+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:56.667+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:56.667+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:56.707+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:56.707+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:56.707+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:57.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:56 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:56.707+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:57 smithi164 podman[73069]: 2024-05-27 07:15:57.064178804 +0000 UTC m=+0.448046944 container init 2bff1c5b5551bd20e57651faafb148fd122b9b3c935a69af8e682067677d90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:57 smithi164 podman[73069]: 2024-05-27 07:15:57.075114797 +0000 UTC m=+0.458982835 container start 2bff1c5b5551bd20e57651faafb148fd122b9b3c935a69af8e682067677d90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:57 smithi164 podman[73069]: 2024-05-27 07:15:57.171330693 +0000 UTC m=+0.555198748 container attach 2bff1c5b5551bd20e57651faafb148fd122b9b3c935a69af8e682067677d90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:57.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:57.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:57.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:57.331 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:57.047+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:15:57.590 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:57.277+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:57.590 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:57.277+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:57.591 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:57.277+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:57.831 INFO:journalctl@ceph.osd.5.smithi164.stdout:May 27 07:15:57 smithi164 podman[73095]: 2024-05-27 07:15:57.593606676 +0000 UTC m=+0.035205536 container died 2bff1c5b5551bd20e57651faafb148fd122b9b3c935a69af8e682067677d90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T07:15:57.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:57.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:57.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:57.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:57.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:57.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:57.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:57.598+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:57.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:57.598+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:57.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:57.598+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:57.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:57 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:57.598+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:57.639+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:57.639+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:57.719+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:57.719+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:57.719+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:58.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:57 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:57.719+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:15:58.325 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:58.089+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:58.325 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:58.089+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:58.325 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:58.089+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:58.325 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:58.089+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:15:58.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T07:15:58.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:15:58.470 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T07:15:58.470 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-27T07:15:58.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.6 2024-05-27T07:15:58.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:58.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:58.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:58.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:58.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:58.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:58.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:58.312+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:15:58.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:58 smithi016 systemd[1]: Stopping Ceph osd.6 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:15:58.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:58.612+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:58.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:58.612+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:58.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:58.612+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:58.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:58.612+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:15:59.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:58.726+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:15:59.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:58.726+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:15:59.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:58 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:58.726+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:15:59.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:58.666+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:15:59.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:58.666+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:15:59.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:58.666+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:15:59.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:58.736+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:15:59.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:58.736+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:15:59.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:58.736+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:15:59.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:58 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:58.736+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:15:59.358 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.063+0000 7fa8dec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:15:59.358 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.063+0000 7fa8dec00700 -1 osd.6 113 *** Got signal Terminated *** 2024-05-27T07:15:59.358 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.063+0000 7fa8dec00700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:15:59.359 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:59.266+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:15:59.359 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:59.266+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:15:59.359 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:59.266+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:15:59.359 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:15:59.266+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:15:59.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:59.138+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:15:59.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:59.138+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:15:59.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:59.138+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:15:59.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:15:59.138+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:15:59.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:15:59.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:15:59.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:15:59.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:15:59 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:15:59.595+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:16:00.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:59.695+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:00.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:59.695+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:00.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:15:59 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:15:59.695+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:00.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:59.688+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:00.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:59.688+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:00.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:15:59.688+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:00.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:59.737+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:00.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:59.737+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:00.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:59.737+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:00.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:15:59 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:15:59.737+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:00.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:00.174+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:00.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:00.174+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:00.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:00.174+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:00.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:00.174+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:00.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:00.550+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:16:00.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:00.550+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:00.550+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:00.550+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:00.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:00.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:00.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:00.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:00 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:00.306+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:00.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:00.679+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:00.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:00.679+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:00.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:00.679+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:00.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:00.725+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:00.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:00.725+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:00.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:00.725+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:00.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:00 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:00.725+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:01.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:00.701+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:01.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:00.701+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:01.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:00 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:00.701+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:01.552 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:01.271+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:01.552 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:01.271+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:01.553 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:01.271+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:01.553 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:01.271+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:01.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:01.158+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:01.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:01.158+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:01.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:01.158+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:01.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:01.158+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:01.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:01.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:16:01.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:01.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:16:01.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:01.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:16:01.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:01 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:01.561+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:16:02.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:01.707+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:02.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:01.707+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:02.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:01 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:01.707+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:01.678+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:01.678+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:01.678+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:01.678+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:01.678+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:01.678+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:02.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:01 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:01.678+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:02.465 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:02.184+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:02.465 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:02.185+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:02.465 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:02.185+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:02.465 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:02.185+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:02.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:02.317+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:02.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:02.317+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:02.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:02.317+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:02.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:02.317+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:02.775 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:02.690+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:02.775 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:02.690+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:02.775 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:02 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:02.690+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:02.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:02.591+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:16:02.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:02.591+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:16:02.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:02.591+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:16:02.858 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:02 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:02.591+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:16:03.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:02.673+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:03.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:02.673+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:03.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:02.673+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:03.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:02.640+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:03.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:02.640+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:03.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:02.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:03.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:02 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:02.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:03.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:03.186+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:03.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:03.186+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:03.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:03.186+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:03.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:03.186+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:03.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:03.323+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:03.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:03.323+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:03.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:03.323+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:03.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:03.323+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:04.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:03.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:04.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:03.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:04.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:03 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:03.718+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:04.108 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:03.630+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:58.823606+0000 front 2024-05-27T07:14:58.824801+0000 (oldest deadline 2024-05-27T07:15:24.123856+0000) 2024-05-27T07:16:04.108 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:03.630+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:10.026717+0000 front 2024-05-27T07:15:10.026920+0000 (oldest deadline 2024-05-27T07:15:38.828313+0000) 2024-05-27T07:16:04.108 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:03.630+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.031673+0000 front 2024-05-27T07:15:24.031964+0000 (oldest deadline 2024-05-27T07:15:49.232949+0000) 2024-05-27T07:16:04.109 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6[57436]: 2024-05-27T07:16:03.630+0000 7fa8d5400700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.535393+0000 front 2024-05-27T07:15:31.535308+0000 (oldest deadline 2024-05-27T07:15:56.835561+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:03.695+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:03.695+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:03.695+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:03.687+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:03.687+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:03.687+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:04.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:03 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:03.687+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:04.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:04.192+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:04.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:04.192+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:04.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:04.192+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:04.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:04.192+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:04.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:04 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:04.347+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:04.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:04 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:04.347+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:04.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:04 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:04.347+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:04.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:04 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:04.347+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:05.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:04.706+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:05.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:04.706+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:05.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:04 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:04.706+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:04.692+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:04.692+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:04.692+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:04.699+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:04.699+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:04.699+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:05.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:04 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:04.699+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:05.358 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:05 smithi016 podman[114314]: 2024-05-27 07:16:05.026671742 +0000 UTC m=+6.109101400 container died 63e813b3d54aa21002a849a998060121241f42c5d6a391d68187a0e5b0a48953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T07:16:05.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:05.184+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:05.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:05.184+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:05.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:05.184+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:05.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:05.184+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:05.740 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:05 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:05.391+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:05.740 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:05 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:05.391+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:05.740 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:05 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:05.391+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:05.740 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:05 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:05.391+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:06.019 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:05 smithi016 podman[114314]: 2024-05-27 07:16:05.749736955 +0000 UTC m=+6.832166614 container cleanup 63e813b3d54aa21002a849a998060121241f42c5d6a391d68187a0e5b0a48953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:16:06.019 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:05 smithi016 bash[114314]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6 2024-05-27T07:16:06.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:05.665+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:06.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:05.665+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:06.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:05 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:05.665+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:05.644+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:05.644+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:05.644+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:05.710+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:05.710+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:05.710+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:06.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:05 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:05.710+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:06.357 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:06 smithi016 podman[114332]: 2024-05-27 07:16:06.026645253 +0000 UTC m=+0.976852103 container remove 63e813b3d54aa21002a849a998060121241f42c5d6a391d68187a0e5b0a48953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T07:16:06.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:06.194+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:06.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:06.194+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:06.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:06.194+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:06.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:06.194+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:06.630 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:06 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:06.380+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:06.630 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:06 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:06.380+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:06.630 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:06 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:06.380+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:06.630 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:06 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:06.380+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:06.615+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:06.616+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:06.616+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:06.703+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:06.703+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:06.703+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:06.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:06 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:06.703+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:06.999 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:06 smithi016 podman[114437]: 2024-05-27 07:16:06.545849488 +0000 UTC m=+0.035116619 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:06.999 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:06 smithi016 podman[114437]: 2024-05-27 07:16:06.70268652 +0000 UTC m=+0.191953681 container create ac0db52679c898dc05c1f50e2c2f04bcca4251f394c7468a5095fc425ba4c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T07:16:06.999 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:07 smithi016 podman[114437]: 2024-05-27 07:16:07.008618537 +0000 UTC m=+0.497885705 container init ac0db52679c898dc05c1f50e2c2f04bcca4251f394c7468a5095fc425ba4c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T07:16:07.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:06.637+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:07.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:06.637+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:07.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:06 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:06.637+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:07.346 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:07 smithi016 podman[114437]: 2024-05-27 07:16:07.020393538 +0000 UTC m=+0.509660706 container start ac0db52679c898dc05c1f50e2c2f04bcca4251f394c7468a5095fc425ba4c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:07.346 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:07 smithi016 podman[114437]: 2024-05-27 07:16:07.121889532 +0000 UTC m=+0.611156733 container attach ac0db52679c898dc05c1f50e2c2f04bcca4251f394c7468a5095fc425ba4c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:07.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:07.216+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:07.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:07.216+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:07.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:07.216+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:07.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:07.216+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:07.608 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 27 07:16:07 smithi016 podman[114437]: 2024-05-27 07:16:07.493509599 +0000 UTC m=+0.982776770 container died ac0db52679c898dc05c1f50e2c2f04bcca4251f394c7468a5095fc425ba4c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T07:16:07.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:07.354+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:07.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:07.354+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:07.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:07.354+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:07.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:07.354+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:07.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:07 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:07.354+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:07.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:07.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:07.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:07.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:07.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:07.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:07.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:07.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:07.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:07.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:07 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:07.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:08.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:07.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:08.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:07.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:08.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:07 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:07.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:08.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.6.service' 2024-05-27T07:16:08.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:08.479 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-27T07:16:08.479 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-27T07:16:08.479 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.7 2024-05-27T07:16:08.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:08.235+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:08.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:08.235+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:08.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:08.235+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:08.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:08.235+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:08.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:08.332+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:08.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:08.332+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:08.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:08.332+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:08.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:08.332+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:08.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:08.332+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:08.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:08.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:08.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:08.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:08.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:08 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:08.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:08.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:08.575+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:08.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:08.575+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:08.867 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:08.575+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:08.575+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:08 smithi069 systemd[1]: Stopping Ceph osd.7 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:08.615+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:08.615+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:08.615+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:08.615+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:08.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:08 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:08.615+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:09.368 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.077+0000 7f1458000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:09.368 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.077+0000 7f1458000700 -1 osd.7 113 *** Got signal Terminated *** 2024-05-27T07:16:09.368 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.077+0000 7f1458000700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:16:09.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:09.236+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:09.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:09.236+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:09.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:09.236+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:09.544 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:09.236+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:09.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:09 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:09.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:09.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:09 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:09.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:09.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:09 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:09.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:09.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:09 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:09.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:09.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:09 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:09.342+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:09.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:09.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:09.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:09.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:09.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:09 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:09.553+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:09.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:09.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:09.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:09.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:09.628+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.586+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:09.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.586+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:09.869 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.586+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:09.869 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:09 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:09.586+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:10.552 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:10.283+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:10.553 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:10.283+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:10.553 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:10.283+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:10.553 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:10.283+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:10.553 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:10.283+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:10.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:10.561+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:10.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:10.561+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:10.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:10.561+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:10.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:10 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:10.561+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:10.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:10 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:10.373+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:10.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:10 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:10.373+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:10.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:10 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:10.373+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:10.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:10 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:10.373+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:10.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:10 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:10.373+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:10.593+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:10.593+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:10.593+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:10.593+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:10.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:10.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:10.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:10.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:10.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:10.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:10 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:10.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:11.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:11.537+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:11.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:11.537+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:11.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:11.537+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:11.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:11.537+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:11.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:11.316+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:11.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:11.316+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:11.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:11.316+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:11.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:11.316+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:11.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:11 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:11.316+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:11.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:11 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:11.366+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:11.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:11 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:11.366+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:11.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:11 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:11.366+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:11.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:11 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:11.366+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:11.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:11 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:11.366+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:11.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:11.596+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:11.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:11.596+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:11.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:11.596+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:11.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:11.596+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:11.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:11.705+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:11.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:11.705+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:11.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:11.705+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:11.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:11.705+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:11.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:11 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:11.705+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:12.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:12.550+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:12.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:12.550+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:12.542 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:12.550+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:12.542 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:12.300+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:12.543 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:12.300+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:12.543 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:12.300+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:12.543 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:12.300+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:12.543 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:12.300+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:12.830 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:12 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:12.550+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:12.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:12 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:12.378+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:12.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:12 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:12.378+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:12.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:12 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:12.378+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:12.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:12 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:12.378+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:12.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:12 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:12.378+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:12.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:12.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:12.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:12.579+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:12.667+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:12.667+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:12.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:12.667+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:12.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:12.667+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:12.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:12 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:12.667+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:13.536 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:13.276+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:13.537 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:13.276+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:13.537 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:13.276+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:13.537 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:13.276+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:13.537 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:13.276+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:13.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:13.545+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:13.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:13.545+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:13.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:13.545+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:13.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:13 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:13.545+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:13.535+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.407200+0000 front 2024-05-27T07:15:14.407081+0000 (oldest deadline 2024-05-27T07:15:39.107397+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:13.535+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.609750+0000 front 2024-05-27T07:15:19.609459+0000 (oldest deadline 2024-05-27T07:15:46.010526+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:13.535+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.913420+0000 front 2024-05-27T07:15:31.912691+0000 (oldest deadline 2024-05-27T07:15:57.713818+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7[53342]: 2024-05-27T07:16:13.535+0000 7f144e800700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.316972+0000 front 2024-05-27T07:15:42.316896+0000 (oldest deadline 2024-05-27T07:16:07.617160+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:13.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:13.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:13.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:13.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:13.843 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:13 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:13.659+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:13.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:13.427+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:13.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:13.427+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:13.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:13.427+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:13.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:13.427+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:13.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:13 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:13.427+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:14.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:14.524+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:14.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:14.524+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:14.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:14.524+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:14.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:14.524+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:14.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:14.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:14.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:14.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:14.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:14.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:14.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:14.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:14.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:14 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:14.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:14.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:14.424+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:14.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:14.424+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:14.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:14.424+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:14.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:14.424+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:14.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:14.424+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:15.085 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:14 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:14.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:15.085 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:14 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:14.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:15.085 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:14 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:14.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:15.085 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:14 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:14.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:15.085 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:14 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:14.657+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:15.367 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:15 smithi069 podman[72865]: 2024-05-27 07:16:15.094422487 +0000 UTC m=+6.163538606 container died ab51c93013e6e26f1ce6f15de3bd982ac3a8c802fb14c63d143030eab2cd981c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:16:15.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:15.554+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:15.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:15.554+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:15.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:15.554+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:15.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:15.554+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:15.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:15.319+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:15.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:15.319+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:15.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:15.319+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:15.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:15.319+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:15.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:15 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:15.319+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:15.751 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:15 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:15.450+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:15.752 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:15 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:15.450+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:15.752 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:15 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:15.450+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:15.752 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:15 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:15.450+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:15.752 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:15 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:15.450+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:15.812 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:15 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:15.641+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:15 smithi069 podman[72865]: 2024-05-27 07:16:15.529441885 +0000 UTC m=+6.598558014 container cleanup ab51c93013e6e26f1ce6f15de3bd982ac3a8c802fb14c63d143030eab2cd981c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:16:15.813 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:15 smithi069 bash[72865]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7 2024-05-27T07:16:16.117 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:15 smithi069 podman[72882]: 2024-05-27 07:16:15.821210328 +0000 UTC m=+0.711487186 container remove ab51c93013e6e26f1ce6f15de3bd982ac3a8c802fb14c63d143030eab2cd981c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:16.523+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:16.523+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:16.523+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:16.523+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:16.318+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:16.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:16.318+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:16.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:16.318+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:16.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:16.318+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:16.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:16 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:16.318+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:16.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:16 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:16.419+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:16.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:16 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:16.419+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:16.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:16 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:16.419+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:16.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:16 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:16.419+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:16.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:16 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:16.419+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:16.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:16 smithi069 podman[72968]: 2024-05-27 07:16:16.507596593 +0000 UTC m=+0.035264695 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:16.868 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:16 smithi069 podman[72968]: 2024-05-27 07:16:16.630977553 +0000 UTC m=+0.158645695 container create 7eb777f9574a1dcbb2ef0cb77523b6b4b2e015eff66ca304f8548b39c430aa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True) 2024-05-27T07:16:16.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:16.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:16.868 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:16.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:16.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:16.869 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:16 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:16.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:17.367 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:16 smithi069 podman[72968]: 2024-05-27 07:16:16.951422926 +0000 UTC m=+0.479091112 container init 7eb777f9574a1dcbb2ef0cb77523b6b4b2e015eff66ca304f8548b39c430aa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:17.368 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:16 smithi069 podman[72968]: 2024-05-27 07:16:16.963134224 +0000 UTC m=+0.490802370 container start 7eb777f9574a1dcbb2ef0cb77523b6b4b2e015eff66ca304f8548b39c430aa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T07:16:17.368 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:17 smithi069 podman[72968]: 2024-05-27 07:16:17.050398309 +0000 UTC m=+0.578066488 container attach 7eb777f9574a1dcbb2ef0cb77523b6b4b2e015eff66ca304f8548b39c430aa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T07:16:17.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:17.536+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:17.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:17.536+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:17.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:17.536+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:17.536+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:17.536+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:17.582 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:17 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:17.304+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 27 07:16:17 smithi069 podman[72968]: 2024-05-27 07:16:17.404473592 +0000 UTC m=+0.932141738 container died 7eb777f9574a1dcbb2ef0cb77523b6b4b2e015eff66ca304f8548b39c430aa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:17.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:17 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:17.637+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:17.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:17 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:17.433+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:17.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:17 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:17.433+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:17.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:17 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:17.433+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:17.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:17 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:17.433+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:17.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:17 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:17.433+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:18.249 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.7.service' 2024-05-27T07:16:18.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:18.357 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-27T07:16:18.357 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-27T07:16:18.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.8 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:18.551+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:18.551+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:18.551+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:18.551+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:18.551+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:18.831 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:18.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:18.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:18.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:18.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.349+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:18.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 systemd[1]: Stopping Ceph osd.8 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:18.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:18 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:18.421+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:18.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:18 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:18.421+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:18.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:18 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:18.421+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:18.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:18 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:18.421+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:18.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:18 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:18.421+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:18.870 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:18.612+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:19.313 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.986+0000 7f1b21600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:19.314 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.986+0000 7f1b21600700 -1 osd.8 113 *** Got signal Terminated *** 2024-05-27T07:16:19.314 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:18 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:18.986+0000 7f1b21600700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:16:19.314 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:19.314 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:19.577 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:19.578 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:19.578 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:19.578 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:19.322+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:19.740 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:19.740 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:19.740 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:19.740 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:19.740 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:19.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:19 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:19.632+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:19.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:19.586+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:19.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:19.586+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:19.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:19.586+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:19.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:19.586+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:19.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:19 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:19.586+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:19.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:19 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:19.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:19.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:19 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:19.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:19.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:19 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:19.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:19.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:19 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:19.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:19.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:19 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:19.428+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:20.565+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:20.565+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:20.565+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:20.565+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:20.565+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:20.831 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:20.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:20.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:20.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:20.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:20.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:20 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:20.358+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:20.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:20.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:20.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:20.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:20.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:20.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:20 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:20.464+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:20.939 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:20.939 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:20.939 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:20.940 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:20.940 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:20.940 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:20 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:20.682+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:21.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:21.548+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:21.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:21.548+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:21.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:21.548+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:21.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:21.548+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:21.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:21.548+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:21.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:21 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:21.374+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:21.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:21.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:21.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:21.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:21.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:21.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:21 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:21.502+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:22.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:22.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:22.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:22.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:22.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:22.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:21 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:21.704+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:22.508+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:22.508+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:22.508+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:22.508+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:22.508+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:22.831 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:22.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:22.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:22.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:22.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:22.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:22 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:22.351+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:22.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:22 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:22.537+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:23.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:22 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:22.712+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:23.477+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:23.477+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:23.477+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:23.477+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:23.477+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:23.831 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:01.069598+0000 front 2024-05-27T07:15:01.069654+0000 (oldest deadline 2024-05-27T07:15:23.369280+0000) 2024-05-27T07:16:23.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:13.274436+0000 front 2024-05-27T07:15:13.274526+0000 (oldest deadline 2024-05-27T07:15:38.574690+0000) 2024-05-27T07:16:23.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:18.576043+0000 front 2024-05-27T07:15:18.576670+0000 (oldest deadline 2024-05-27T07:15:49.077979+0000) 2024-05-27T07:16:23.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:31.980710+0000 front 2024-05-27T07:15:31.981100+0000 (oldest deadline 2024-05-27T07:15:55.480633+0000) 2024-05-27T07:16:23.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:42.886546+0000 front 2024-05-27T07:15:42.885993+0000 (oldest deadline 2024-05-27T07:16:09.287382+0000) 2024-05-27T07:16:23.832 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:23 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8[53534]: 2024-05-27T07:16:23.371+0000 7f1b17e00700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.789517+0000 front 2024-05-27T07:15:52.789423+0000 (oldest deadline 2024-05-27T07:16:16.889703+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:23.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:23 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:23.586+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:24.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:23 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:23.753+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:24.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:24.455+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:24.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:24.455+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:24.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:24.455+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:24.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:24.455+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:24.782 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:24 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:24.455+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:24.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:24.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:24.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:24.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:24.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:24.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:24 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:24.540+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:25.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:24 smithi164 podman[73188]: 2024-05-27 07:16:24.791677158 +0000 UTC m=+5.955173424 container died c1a4b7f7106a1b6fff6db0ac9b29d695ef792fc67900410ebccaed0542467ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T07:16:25.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:25.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:25.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:25.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:25.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:25.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:24 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:24.721+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:25.472 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:25 smithi164 podman[73188]: 2024-05-27 07:16:25.328100204 +0000 UTC m=+6.491596500 container cleanup c1a4b7f7106a1b6fff6db0ac9b29d695ef792fc67900410ebccaed0542467ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T07:16:25.472 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:25 smithi164 bash[73188]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8 2024-05-27T07:16:25.830 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:25 smithi164 podman[73204]: 2024-05-27 07:16:25.538069006 +0000 UTC m=+0.736109209 container remove c1a4b7f7106a1b6fff6db0ac9b29d695ef792fc67900410ebccaed0542467ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T07:16:25.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:25.481+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:25.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:25.481+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:25.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:25.481+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:25.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:25.481+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:25.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:25 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:25.481+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:25.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:25.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:25.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:25.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:25.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:25.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:25 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:25.572+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:26.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:26.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:26.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:26.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:26.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:26.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:25 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:25.757+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:26.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:26 smithi164 podman[73291]: 2024-05-27 07:16:26.33062629 +0000 UTC m=+0.122048042 container create a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:26.581 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:26 smithi164 podman[73291]: 2024-05-27 07:16:26.240685226 +0000 UTC m=+0.032106941 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:26.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:26.581 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:26.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:26.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:26.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:26.582 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:26 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:26.452+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:26.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:26.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:26.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:26.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:26.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:26.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:26 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:26.543+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:27.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:26 smithi164 podman[73291]: 2024-05-27 07:16:26.667923069 +0000 UTC m=+0.459344831 container init a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T07:16:27.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:26 smithi164 podman[73291]: 2024-05-27 07:16:26.6791251 +0000 UTC m=+0.470546859 container start a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:27.081 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:26 smithi164 podman[73291]: 2024-05-27 07:16:26.841931748 +0000 UTC m=+0.633353557 container attach a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T07:16:27.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.782+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:27.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.782+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:27.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.782+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:27.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.782+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:27.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.782+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:27.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:26 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:26.783+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:27.421 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:27 smithi164 podman[73291]: 2024-05-27 07:16:27.13604881 +0000 UTC m=+0.927470568 container died a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T07:16:27.813 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:27.814 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:27.814 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:27.814 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:27.814 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:27.814 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:27 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:27.430+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:27.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:27.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:27.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:27.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:27.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:27.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:27 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:27.546+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:28.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.8.service' 2024-05-27T07:16:28.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:28 smithi164 podman[73317]: 2024-05-27 07:16:28.021538564 +0000 UTC m=+0.865963889 container remove a455c7f8e5e695c5f647d4ae4cc66ca5bb20074ea4106a759cad2c2605955df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:16:28.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:28 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.8.service: Deactivated successfully. 2024-05-27T07:16:28.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:28 smithi164 systemd[1]: Stopped Ceph osd.8 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968. 2024-05-27T07:16:28.080 INFO:journalctl@ceph.osd.8.smithi164.stdout:May 27 07:16:28 smithi164 systemd[1]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.8.service: Consumed 8min 2.138s CPU time. 2024-05-27T07:16:28.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:28.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:28.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:28.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:28.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:28.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:27.833+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:28.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:28.142 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-27T07:16:28.142 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-27T07:16:28.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.9 2024-05-27T07:16:28.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 systemd[1]: Stopping Ceph osd.9 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:28.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:28.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:28.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:28.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:28.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:28.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:28 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:28.439+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:28.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.591+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.752+0000 7f3e90000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.752+0000 7f3e90000700 -1 osd.9 113 *** Got signal Terminated *** 2024-05-27T07:16:28.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:28 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:28.752+0000 7f3e90000700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:16:29.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:29.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:28 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:28.828+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:29.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:29 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:29.438+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:29.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:29 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:29.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:30.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:30.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:29 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:29.810+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:30.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:30 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:30.414+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:30.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:30.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:30.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:30.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:30.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:30.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:30 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:30.596+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:31.067 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:30 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:30.797+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:31.733 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:31 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:31.368+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:32.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:31 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:31.635+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:32.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:32.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:32.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:32.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:32.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:32.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:32.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:31 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:31.765+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:32.749 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:32.749 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:32.749 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:32.749 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:32.750 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:32.750 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:32 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:32.401+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:33.010 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:33.010 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:33.010 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:33.010 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:33.010 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:33.011 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:32 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:32.654+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:33.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:33.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:32 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:32.748+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:33.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:33 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:33.416+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:34.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.970746+0000 front 2024-05-27T07:14:57.970550+0000 (oldest deadline 2024-05-27T07:15:23.870422+0000) 2024-05-27T07:16:34.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.374620+0000 front 2024-05-27T07:15:14.374681+0000 (oldest deadline 2024-05-27T07:15:39.674507+0000) 2024-05-27T07:16:34.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:19.675839+0000 front 2024-05-27T07:15:19.675698+0000 (oldest deadline 2024-05-27T07:15:46.077253+0000) 2024-05-27T07:16:34.109 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:34.183170+0000 front 2024-05-27T07:15:34.182664+0000 (oldest deadline 2024-05-27T07:15:58.282667+0000) 2024-05-27T07:16:34.109 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:43.486079+0000 front 2024-05-27T07:15:43.485917+0000 (oldest deadline 2024-05-27T07:16:06.985236+0000) 2024-05-27T07:16:34.109 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:33 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9[64297]: 2024-05-27T07:16:33.698+0000 7f3e8b800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.988301+0000 front 2024-05-27T07:15:53.988378+0000 (oldest deadline 2024-05-27T07:16:19.887836+0000) 2024-05-27T07:16:34.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:34.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:34.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:34.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:34.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:34.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:34.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:33 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:33.720+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:34.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:34 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:34.467+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:35.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:34 smithi016 podman[114541]: 2024-05-27 07:16:34.818423941 +0000 UTC m=+6.198655700 container died 85fbc6061b844ff67f2da8b4eaec252e00029da509437b01ee1b1884e3c7528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521) 2024-05-27T07:16:35.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:35.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:34 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:34.727+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:35.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:35 smithi016 podman[114541]: 2024-05-27 07:16:35.369744279 +0000 UTC m=+6.749976070 container cleanup 85fbc6061b844ff67f2da8b4eaec252e00029da509437b01ee1b1884e3c7528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T07:16:35.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:35 smithi016 bash[114541]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:35.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:35 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:35.419+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:36.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:35 smithi016 podman[114571]: 2024-05-27 07:16:35.663517619 +0000 UTC m=+0.835372417 container remove 85fbc6061b844ff67f2da8b4eaec252e00029da509437b01ee1b1884e3c7528b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.759+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:36.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:35.760+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:36.809 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:36 smithi016 podman[114656]: 2024-05-27 07:16:36.420660931 +0000 UTC m=+0.038367356 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:36.809 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:36 smithi016 podman[114656]: 2024-05-27 07:16:36.53079642 +0000 UTC m=+0.148502876 container create 390b9775bf3e3063c06001d00ac7182ffce9969e4cce141cd836117b147744fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T07:16:36.809 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:36 smithi016 podman[114656]: 2024-05-27 07:16:36.817811301 +0000 UTC m=+0.435517737 container init 390b9775bf3e3063c06001d00ac7182ffce9969e4cce141cd836117b147744fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:36.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:36 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:36.454+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:37.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:36 smithi016 podman[114656]: 2024-05-27 07:16:36.830912173 +0000 UTC m=+0.448618659 container start 390b9775bf3e3063c06001d00ac7182ffce9969e4cce141cd836117b147744fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T07:16:37.108 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:36 smithi016 podman[114656]: 2024-05-27 07:16:36.916528312 +0000 UTC m=+0.534234828 container attach 390b9775bf3e3063c06001d00ac7182ffce9969e4cce141cd836117b147744fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T07:16:37.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.807+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:37.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.807+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.807+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.807+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.808+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.808+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.808+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:37.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:36 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:36.808+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:37.608 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 27 07:16:37 smithi016 podman[114656]: 2024-05-27 07:16:37.321604062 +0000 UTC m=+0.939310558 container died 390b9775bf3e3063c06001d00ac7182ffce9969e4cce141cd836117b147744fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:37.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:37 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:37.420+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:38.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.9.service' 2024-05-27T07:16:38.117 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:38.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:37 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:37.843+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:38.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:38.223 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-27T07:16:38.223 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-27T07:16:38.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.10 2024-05-27T07:16:38.617 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 systemd[1]: Stopping Ceph osd.10 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:38.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:38 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:38.464+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:39.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:39.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:39.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.825+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:39.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.865+0000 7fc6ddc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:39.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.865+0000 7fc6ddc00700 -1 osd.10 113 *** Got signal Terminated *** 2024-05-27T07:16:39.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:38 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:38.865+0000 7fc6ddc00700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:39.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:39 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:39.486+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:40.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:39 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:39.851+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:40.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:40 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:40.522+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:41.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:40 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:40.852+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:41.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:41 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:41.555+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:42.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:42.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:42.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:42.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:42.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:42.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:42.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:42.119 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:41 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:41.857+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:42.885 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:42.885 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:42.886 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:42.886 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:42.886 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:42.886 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:42.886 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:42 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:42.596+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:14:57.262411+0000 front 2024-05-27T07:14:57.262671+0000 (oldest deadline 2024-05-27T07:15:28.463225+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:14.267481+0000 front 2024-05-27T07:15:14.267151+0000 (oldest deadline 2024-05-27T07:15:40.167000+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.268738+0000 front 2024-05-27T07:15:24.268641+0000 (oldest deadline 2024-05-27T07:15:45.968881+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-27T07:15:30.975791+0000 front 2024-05-27T07:15:30.975601+0000 (oldest deadline 2024-05-27T07:15:55.075823+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.281671+0000 front 2024-05-27T07:15:44.281598+0000 (oldest deadline 2024-05-27T07:16:07.181792+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:52.385579+0000 front 2024-05-27T07:15:52.385446+0000 (oldest deadline 2024-05-27T07:16:15.285510+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:16:03.989387+0000 front 2024-05-27T07:16:03.988998+0000 (oldest deadline 2024-05-27T07:16:28.088696+0000) 2024-05-27T07:16:43.118 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10[60136]: 2024-05-27T07:16:42.896+0000 7fc6d9400700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:13.390808+0000 front 2024-05-27T07:16:13.390887+0000 (oldest deadline 2024-05-27T07:16:35.090747+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:43.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:43.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:43 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:43.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:44.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:44 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:44.577+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:44.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:44 smithi069 podman[73097]: 2024-05-27 07:16:44.609402661 +0000 UTC m=+5.891279152 container died e7c00c289edd5aa44dedafe0ef31a35831f87ec9cf18a8eeb0a6e04c90c91da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:16:45.315 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:45 smithi069 podman[73097]: 2024-05-27 07:16:45.090713922 +0000 UTC m=+6.372590377 container cleanup e7c00c289edd5aa44dedafe0ef31a35831f87ec9cf18a8eeb0a6e04c90c91da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:16:45.316 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:45 smithi069 bash[73097]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10 2024-05-27T07:16:45.617 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:45 smithi069 podman[73116]: 2024-05-27 07:16:45.325144678 +0000 UTC m=+0.677667870 container remove e7c00c289edd5aa44dedafe0ef31a35831f87ec9cf18a8eeb0a6e04c90c91da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:45.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:45 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:45.584+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:46.446 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73205]: 2024-05-27 07:16:46.086672726 +0000 UTC m=+0.032493266 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:46.446 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73205]: 2024-05-27 07:16:46.184338558 +0000 UTC m=+0.130159134 container create 4d93dff55b7b9fa569e729c1791ceb2e7d8fbf6a38d87b3624fb07f0c9cfe615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T07:16:46.446 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73205]: 2024-05-27 07:16:46.455526172 +0000 UTC m=+0.401346755 container init 4d93dff55b7b9fa569e729c1791ceb2e7d8fbf6a38d87b3624fb07f0c9cfe615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:46.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:46 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:46.578+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:46.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73205]: 2024-05-27 07:16:46.466376173 +0000 UTC m=+0.412196735 container start 4d93dff55b7b9fa569e729c1791ceb2e7d8fbf6a38d87b3624fb07f0c9cfe615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-27T07:16:46.867 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73205]: 2024-05-27 07:16:46.562848541 +0000 UTC m=+0.508669127 container attach 4d93dff55b7b9fa569e729c1791ceb2e7d8fbf6a38d87b3624fb07f0c9cfe615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-05-27T07:16:47.236 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 27 07:16:46 smithi069 podman[73231]: 2024-05-27 07:16:46.983169092 +0000 UTC m=+0.038110012 container died 4d93dff55b7b9fa569e729c1791ceb2e7d8fbf6a38d87b3624fb07f0c9cfe615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T07:16:47.622 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.10.service' 2024-05-27T07:16:47.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:47.737 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-27T07:16:47.737 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-27T07:16:47.737 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.11 2024-05-27T07:16:47.815 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:47.816 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:47.816 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:47.816 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:47.816 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:47.816 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:47.817 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:47.533+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:48.081 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:47 smithi164 systemd[1]: Stopping Ceph osd.11 for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.437+0000 7feb19a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.437+0000 7feb19a00700 -1 osd.11 113 *** Got signal Terminated *** 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.437+0000 7feb19a00700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:48.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:48.833 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:48.833 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:48.833 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:48.833 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:48.573+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-05-27T07:16:22.547004+0000 front 2024-05-27T07:16:22.547069+0000 (oldest deadline 2024-05-27T07:16:47.846863+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:49.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:49 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:49.530+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-05-27T07:16:22.547004+0000 front 2024-05-27T07:16:22.547069+0000 (oldest deadline 2024-05-27T07:16:47.846863+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:50.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:50.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:50 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:50.492+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-05-27T07:16:22.547004+0000 front 2024-05-27T07:16:22.547069+0000 (oldest deadline 2024-05-27T07:16:47.846863+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:51.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:51.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:51.453+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-05-27T07:16:22.547004+0000 front 2024-05-27T07:16:22.547069+0000 (oldest deadline 2024-05-27T07:16:47.846863+0000) 2024-05-27T07:16:52.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-27T07:15:00.922620+0000 front 2024-05-27T07:15:00.922380+0000 (oldest deadline 2024-05-27T07:15:23.222145+0000) 2024-05-27T07:16:52.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-27T07:15:11.425933+0000 front 2024-05-27T07:15:11.426541+0000 (oldest deadline 2024-05-27T07:15:36.725830+0000) 2024-05-27T07:16:52.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-05-27T07:15:24.230697+0000 front 2024-05-27T07:15:24.231666+0000 (oldest deadline 2024-05-27T07:15:50.130520+0000) 2024-05-27T07:16:52.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-27T07:15:44.234984+0000 front 2024-05-27T07:15:44.235489+0000 (oldest deadline 2024-05-27T07:16:10.134819+0000) 2024-05-27T07:16:52.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-05-27T07:15:53.438897+0000 front 2024-05-27T07:15:53.438999+0000 (oldest deadline 2024-05-27T07:16:16.939084+0000) 2024-05-27T07:16:52.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-27T07:15:59.742099+0000 front 2024-05-27T07:15:59.742191+0000 (oldest deadline 2024-05-27T07:16:25.642174+0000) 2024-05-27T07:16:52.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-27T07:16:10.944097+0000 front 2024-05-27T07:16:10.944191+0000 (oldest deadline 2024-05-27T07:16:36.144645+0000) 2024-05-27T07:16:52.832 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:52 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11[60184]: 2024-05-27T07:16:52.479+0000 7feb15200700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-05-27T07:16:22.547004+0000 front 2024-05-27T07:16:22.547069+0000 (oldest deadline 2024-05-27T07:16:47.846863+0000) 2024-05-27T07:16:54.572 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:54 smithi164 podman[73409]: 2024-05-27 07:16:54.244889341 +0000 UTC m=+6.027725223 container died 0c16bdb0026d54d1d6256387a391d00ea60932984d8966095b17f7e1b462208b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T07:16:54.830 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:54 smithi164 podman[73409]: 2024-05-27 07:16:54.706651482 +0000 UTC m=+6.489487365 container cleanup 0c16bdb0026d54d1d6256387a391d00ea60932984d8966095b17f7e1b462208b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T07:16:54.830 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:54 smithi164 bash[73409]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11 2024-05-27T07:16:55.331 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:54 smithi164 podman[73428]: 2024-05-27 07:16:54.949601426 +0000 UTC m=+0.684196404 container remove 0c16bdb0026d54d1d6256387a391d00ea60932984d8966095b17f7e1b462208b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T07:16:56.022 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:55 smithi164 podman[73512]: 2024-05-27 07:16:55.679779301 +0000 UTC m=+0.037380544 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:16:56.023 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:55 smithi164 podman[73512]: 2024-05-27 07:16:55.775246517 +0000 UTC m=+0.132847790 container create ee4c9bf13f5b0a55f35a376e73256d293425933d4619f2127b44683082a9190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T07:16:56.023 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:56 smithi164 podman[73512]: 2024-05-27 07:16:56.020741974 +0000 UTC m=+0.378343316 container init ee4c9bf13f5b0a55f35a376e73256d293425933d4619f2127b44683082a9190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T07:16:56.023 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:56 smithi164 podman[73512]: 2024-05-27 07:16:56.032017395 +0000 UTC m=+0.389618684 container start ee4c9bf13f5b0a55f35a376e73256d293425933d4619f2127b44683082a9190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:56.330 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:56 smithi164 podman[73512]: 2024-05-27 07:16:56.136243911 +0000 UTC m=+0.493845220 container attach ee4c9bf13f5b0a55f35a376e73256d293425933d4619f2127b44683082a9190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T07:16:56.831 INFO:journalctl@ceph.osd.11.smithi164.stdout:May 27 07:16:56 smithi164 podman[73512]: 2024-05-27 07:16:56.471184794 +0000 UTC m=+0.828786065 container died ee4c9bf13f5b0a55f35a376e73256d293425933d4619f2127b44683082a9190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T07:16:57.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@osd.11.service' 2024-05-27T07:16:57.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:16:57.401 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-27T07:16:57.401 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-27T07:16:57.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.a 2024-05-27T07:16:57.858 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:16:57 smithi016 systemd[1]: Stopping Ceph mds.a for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:16:58.358 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:16:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a[71684]: 2024-05-27T07:16:58.019+0000 7fd308c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:58.358 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:16:58 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a[71684]: 2024-05-27T07:16:58.019+0000 7fd308c00700 -1 mds.a *** got signal Terminated *** 2024-05-27T07:17:01.417 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:17:01 smithi016 podman[114761]: 2024-05-27 07:17:01.14467237 +0000 UTC m=+3.238523158 container died d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T07:17:01.858 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:17:01 smithi016 podman[114761]: 2024-05-27 07:17:01.654947524 +0000 UTC m=+3.748798269 container cleanup d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-05-27T07:17:01.858 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:17:01 smithi016 bash[114761]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a 2024-05-27T07:17:02.357 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 27 07:17:01 smithi016 podman[114773]: 2024-05-27 07:17:01.973123564 +0000 UTC m=+0.808340352 container remove d5b3822c5947fb7ed9ae33bb43117b6f30bbab83c1a39993edaf2c87f4708664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:17:02.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.a.service' 2024-05-27T07:17:02.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:02.723 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-27T07:17:02.723 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-27T07:17:02.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d 2024-05-27T07:17:02.857 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:02 smithi016 systemd[1]: Stopping Ceph mds.d for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:03.608 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[112061]: 2024-05-27T07:17:03.314+0000 7ff761400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:03.608 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:03 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d[112061]: 2024-05-27T07:17:03.314+0000 7ff761400700 -1 mds.d *** got signal Terminated *** 2024-05-27T07:17:06.955 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:06 smithi016 podman[114880]: 2024-05-27 07:17:06.641984527 +0000 UTC m=+3.456824562 container stop 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True) 2024-05-27T07:17:06.955 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:06 smithi016 podman[114880]: 2024-05-27 07:17:06.670521751 +0000 UTC m=+3.485361755 container died 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T07:17:07.358 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:07 smithi016 podman[114880]: 2024-05-27 07:17:07.175996998 +0000 UTC m=+3.990837030 container cleanup 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2) 2024-05-27T07:17:07.358 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:07 smithi016 bash[114880]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d 2024-05-27T07:17:07.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 27 07:17:07 smithi016 podman[114894]: 2024-05-27 07:17:07.435834301 +0000 UTC m=+0.772006385 container remove 4ed235f9f67689611897eb39899061974c09892ce72de0c7e051cbeaea236e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T07:17:07.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.d.service' 2024-05-27T07:17:08.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:08.065 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-27T07:17:08.066 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-27T07:17:08.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.g 2024-05-27T07:17:08.608 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:08 smithi016 systemd[1]: Stopping Ceph mds.g for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:09.108 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g[71287]: 2024-05-27T07:17:08.702+0000 7f4305c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:09.108 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:08 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g[71287]: 2024-05-27T07:17:08.702+0000 7f4305c00700 -1 mds.g *** got signal Terminated *** 2024-05-27T07:17:12.458 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:12 smithi016 podman[114999]: 2024-05-27 07:17:12.1816114 +0000 UTC m=+3.620844645 container died 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:17:13.108 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:12 smithi016 podman[114999]: 2024-05-27 07:17:12.736216492 +0000 UTC m=+4.175449664 container cleanup 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:17:13.109 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:12 smithi016 bash[114999]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g 2024-05-27T07:17:13.109 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 27 07:17:12 smithi016 podman[115012]: 2024-05-27 07:17:12.929399003 +0000 UTC m=+0.726845429 container remove 68d541561a766462b31752f5e885ad91f7517518bc2926b297aae61a5eddd4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T07:17:13.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.g.service' 2024-05-27T07:17:13.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:13.514 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-27T07:17:13.514 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-27T07:17:13.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.j 2024-05-27T07:17:13.858 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:13 smithi016 systemd[1]: Stopping Ceph mds.j for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:14.358 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j[69932]: 2024-05-27T07:17:14.103+0000 7f2a1f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:14.358 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:14 smithi016 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j[69932]: 2024-05-27T07:17:14.103+0000 7f2a1f800700 -1 mds.j *** got signal Terminated *** 2024-05-27T07:17:17.055 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:16 smithi016 podman[115120]: 2024-05-27 07:17:16.787070009 +0000 UTC m=+2.806084474 container died 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:17:17.358 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:17 smithi016 podman[115120]: 2024-05-27 07:17:17.173995541 +0000 UTC m=+3.193010002 container cleanup 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T07:17:17.358 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:17 smithi016 bash[115120]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j 2024-05-27T07:17:17.858 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 27 07:17:17 smithi016 podman[115135]: 2024-05-27 07:17:17.483247249 +0000 UTC m=+0.683427234 container remove 1fa8905d0eff586364b1975530d410cd2e0fd0d0cceadcef9c193ebf801addc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T07:17:17.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.j.service' 2024-05-27T07:17:18.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:18.094 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-27T07:17:18.094 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-27T07:17:18.094 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.b 2024-05-27T07:17:18.502 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:18 smithi069 systemd[1]: Stopping Ceph mds.b for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:18.867 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T07:17:18.654+0000 7f0aca400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:18.867 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:18 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b[65474]: 2024-05-27T07:17:18.655+0000 7f0aca400700 -1 mds.b *** got signal Terminated *** 2024-05-27T07:17:25.985 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:25 smithi069 podman[73309]: 2024-05-27 07:17:25.651073317 +0000 UTC m=+7.116098187 container died 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T07:17:26.299 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:26 smithi069 podman[73309]: 2024-05-27 07:17:26.115838734 +0000 UTC m=+7.580863558 container cleanup 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T07:17:26.299 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:26 smithi069 bash[73309]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b 2024-05-27T07:17:26.617 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 27 07:17:26 smithi069 podman[73334]: 2024-05-27 07:17:26.30855724 +0000 UTC m=+0.617038169 container remove 3cb4a848d3440be4ddb1c482912715ea52294b8d119575e43e6a4903486dcb9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T07:17:26.851 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.b.service' 2024-05-27T07:17:26.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:26.968 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-27T07:17:26.968 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-27T07:17:26.968 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.e 2024-05-27T07:17:27.367 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:27 smithi069 systemd[1]: Stopping Ceph mds.e for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:27.867 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e[64861]: 2024-05-27T07:17:27.585+0000 7fef2aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:27.868 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:27 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e[64861]: 2024-05-27T07:17:27.585+0000 7fef2aa00700 -1 mds.e *** got signal Terminated *** 2024-05-27T07:17:33.910 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:33 smithi069 podman[73440]: 2024-05-27 07:17:33.65519045 +0000 UTC m=+6.229648103 container died cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T07:17:34.244 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:34 smithi069 podman[73440]: 2024-05-27 07:17:34.036029965 +0000 UTC m=+6.610487601 container cleanup cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:17:34.245 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:34 smithi069 bash[73440]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e 2024-05-27T07:17:34.618 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 27 07:17:34 smithi069 podman[73453]: 2024-05-27 07:17:34.253903124 +0000 UTC m=+0.577222290 container remove cece82aadf2aabdae93fc75ad0570df30b7bc16e6d1a6070e937d63fcb2bc6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T07:17:34.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.e.service' 2024-05-27T07:17:34.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:34.827 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-27T07:17:34.827 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-27T07:17:34.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.h 2024-05-27T07:17:35.367 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:34 smithi069 systemd[1]: Stopping Ceph mds.h for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:35.867 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h[65171]: 2024-05-27T07:17:35.421+0000 7f736da00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:35.867 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:35 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h[65171]: 2024-05-27T07:17:35.422+0000 7f736da00700 -1 mds.h *** got signal Terminated *** 2024-05-27T07:17:41.051 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:40 smithi069 podman[73561]: 2024-05-27 07:17:40.740173341 +0000 UTC m=+5.436967932 container died c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T07:17:41.356 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:41 smithi069 podman[73561]: 2024-05-27 07:17:41.181064629 +0000 UTC m=+5.877859203 container cleanup c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:17:41.356 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:41 smithi069 bash[73561]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h 2024-05-27T07:17:41.617 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 27 07:17:41 smithi069 podman[73574]: 2024-05-27 07:17:41.365465086 +0000 UTC m=+0.610253300 container remove c69f7a53817c18a2915a0d024be5539dc9aa9ae58792530fc73cfa8d03e8756f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T07:17:41.832 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.h.service' 2024-05-27T07:17:41.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:41.927 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-27T07:17:41.927 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-27T07:17:41.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.k 2024-05-27T07:17:42.366 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:42 smithi069 systemd[1]: Stopping Ceph mds.k for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:42.867 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k[64560]: 2024-05-27T07:17:42.531+0000 7fde7aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:42.867 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:42 smithi069 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k[64560]: 2024-05-27T07:17:42.531+0000 7fde7aa00700 -1 mds.k *** got signal Terminated *** 2024-05-27T07:17:46.867 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:46 smithi069 podman[73680]: 2024-05-27 07:17:46.599022996 +0000 UTC m=+4.212271721 container died 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T07:17:47.226 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:47 smithi069 podman[73680]: 2024-05-27 07:17:47.026141155 +0000 UTC m=+4.639389849 container cleanup 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD) 2024-05-27T07:17:47.226 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:47 smithi069 bash[73680]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k 2024-05-27T07:17:47.617 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 27 07:17:47 smithi069 podman[73694]: 2024-05-27 07:17:47.235626692 +0000 UTC m=+0.594116128 container remove 9fe235a28b524f05d5a7c31e94cb4ad7b1d710ee684548024fa3a499c3270ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T07:17:47.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.k.service' 2024-05-27T07:17:47.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:47.889 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-27T07:17:47.889 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-27T07:17:47.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c 2024-05-27T07:17:48.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.c.service' 2024-05-27T07:17:48.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:48.104 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-27T07:17:48.104 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-27T07:17:48.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.f 2024-05-27T07:17:48.330 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:48 smithi164 systemd[1]: Stopping Ceph mds.f for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:48.764 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f[64722]: 2024-05-27T07:17:48.601+0000 7f81f2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:48.765 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:48 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f[64722]: 2024-05-27T07:17:48.601+0000 7f81f2400700 -1 mds.f *** got signal Terminated *** 2024-05-27T07:17:50.371 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:50 smithi164 podman[73653]: 2024-05-27 07:17:50.091136868 +0000 UTC m=+1.642469794 container died a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:17:50.788 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:50 smithi164 podman[73653]: 2024-05-27 07:17:50.496534247 +0000 UTC m=+2.047867167 container cleanup a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-27T07:17:50.788 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:50 smithi164 bash[73653]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f 2024-05-27T07:17:51.059 INFO:journalctl@ceph.mds.f.smithi164.stdout:May 27 07:17:50 smithi164 podman[73675]: 2024-05-27 07:17:50.797434648 +0000 UTC m=+0.692075763 container remove a66e6e055a65292a8dbdee2767de1bf97b426f088bdc28a7cc53e4f8a37fe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:17:51.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.f.service' 2024-05-27T07:17:51.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:51.202 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-27T07:17:51.202 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-27T07:17:51.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.i 2024-05-27T07:17:51.798 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:51 smithi164 systemd[1]: Stopping Ceph mds.i for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:52.080 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i[65331]: 2024-05-27T07:17:51.807+0000 7f02cd600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:52.080 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:51 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i[65331]: 2024-05-27T07:17:51.807+0000 7f02cd600700 -1 mds.i *** got signal Terminated *** 2024-05-27T07:17:56.080 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:55 smithi164 podman[73784]: 2024-05-27 07:17:55.827169984 +0000 UTC m=+4.150304944 container died 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:17:56.525 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:56 smithi164 podman[73784]: 2024-05-27 07:17:56.300074646 +0000 UTC m=+4.623209600 container cleanup 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T07:17:56.525 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:56 smithi164 bash[73784]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i 2024-05-27T07:17:56.830 INFO:journalctl@ceph.mds.i.smithi164.stdout:May 27 07:17:56 smithi164 podman[73799]: 2024-05-27 07:17:56.534519208 +0000 UTC m=+0.688677988 container remove 4523bd94ae83250132beb06277bd99eaaf7b137dff18299138bb15611fcdfb16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T07:17:57.001 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.i.service' 2024-05-27T07:17:57.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:17:57.109 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-27T07:17:57.109 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-27T07:17:57.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.l 2024-05-27T07:17:57.330 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:57 smithi164 systemd[1]: Stopping Ceph mds.l for 3c4673a4-1bf4-11ef-bc9b-c7b262605968... 2024-05-27T07:17:58.080 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l[64421]: 2024-05-27T07:17:57.659+0000 7f9822400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:17:58.080 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:57 smithi164 ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l[64421]: 2024-05-27T07:17:57.659+0000 7f9822400700 -1 mds.l *** got signal Terminated *** 2024-05-27T07:17:59.580 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:59 smithi164 podman[73907]: 2024-05-27 07:17:59.278292758 +0000 UTC m=+1.743196888 container died f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:17:59.982 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:59 smithi164 podman[73907]: 2024-05-27 07:17:59.706928798 +0000 UTC m=+2.171832888 container cleanup f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T07:17:59.982 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:59 smithi164 bash[73907]: ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l 2024-05-27T07:18:00.330 INFO:journalctl@ceph.mds.l.smithi164.stdout:May 27 07:17:59 smithi164 podman[73919]: 2024-05-27 07:17:59.991047502 +0000 UTC m=+0.687016328 container remove f3738bac5aee39f43ad9b2ce3d99d3265e0890a8f4ca42bd3efded6abce7de47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:18:00.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c4673a4-1bf4-11ef-bc9b-c7b262605968@mds.l.service' 2024-05-27T07:18:00.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T07:18:00.623 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-27T07:18:00.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T07:18:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:18:06.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T07:18:06.724 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:18:11.766 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T07:18:12.080 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:18:17.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:17.254 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:17.347 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:17.439 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T07:18:17.442 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi016/crash 2024-05-27T07:18:17.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T07:18:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T07:18:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-05-27T07:18:17.560 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi069/crash 2024-05-27T07:18:17.561 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T07:18:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T07:18:17.666 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-05-27T07:18:17.671 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi164/crash 2024-05-27T07:18:17.672 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T07:18:17.775 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T07:18:17.776 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-05-27T07:18:17.779 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T07:18:17.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-27T07:18:17.955 INFO:tasks.cephadm:Compressing logs... 2024-05-27T07:18:17.955 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T07:18:17.960 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T07:18:17.968 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T07:18:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T07:18:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T07:18:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T07:18:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-05-27T07:18:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-05-27T07:18:18.061 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T07:18:18.061 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T07:18:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T07:18:18.062 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T07:18:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T07:18:18.063 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-05-27T07:18:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-05-27T07:18:18.064 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-05-27T07:18:18.065 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T07:18:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-05-27T07:18:18.066 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T07:18:18.067 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T07:18:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-05-27T07:18:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-05-27T07:18:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T07:18:18.068 INFO:teuthology.orchestra.run.smithi016.stderr: 94.3% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-05-27T07:18:18.069 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-05-27T07:18:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-05-27T07:18:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-05-27T07:18:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T07:18:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.h.log: 87.4% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T07:18:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T07:18:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-05-27T07:18:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.10.log: -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-05-27T07:18:18.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-05-27T07:18:18.073 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T07:18:18.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-27T07:18:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-05-27T07:18:18.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T07:18:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log: 83.6% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T07:18:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-05-27T07:18:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-27T07:18:18.076 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T07:18:18.076 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-05-27T07:18:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T07:18:18.077 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T07:18:18.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-05-27T07:18:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-05-27T07:18:18.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T07:18:18.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T07:18:18.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-05-27T07:18:18.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-05-27T07:18:18.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-05-27T07:18:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-05-27T07:18:18.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T07:18:18.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-05-27T07:18:18.089 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-05-27T07:18:18.090 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T07:18:18.091 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-05-27T07:18:18.092 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T07:18:18.096 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-05-27T07:18:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 93.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-05-27T07:18:18.163 INFO:teuthology.orchestra.run.smithi069.stderr: 90.3% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T07:18:18.200 INFO:teuthology.orchestra.run.smithi016.stderr: 84.0% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T07:18:18.209 INFO:teuthology.orchestra.run.smithi069.stderr: 93.5% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-05-27T07:18:18.213 INFO:teuthology.orchestra.run.smithi069.stderr: 93.5% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-05-27T07:18:18.220 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-05-27T07:18:18.240 INFO:teuthology.orchestra.run.smithi069.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T07:18:18.245 INFO:teuthology.orchestra.run.smithi164.stderr: 83.5% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T07:18:18.248 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T07:18:18.258 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T07:18:18.272 INFO:teuthology.orchestra.run.smithi164.stderr: 90.3% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T07:18:18.284 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T07:18:18.363 INFO:teuthology.orchestra.run.smithi164.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T07:18:18.364 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-27T07:18:18.370 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T07:18:18.383 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T07:18:18.446 INFO:teuthology.orchestra.run.smithi164.stderr: 95.0% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-05-27T07:18:18.453 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-05-27T07:18:18.460 INFO:teuthology.orchestra.run.smithi164.stderr: 94.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-05-27T07:18:18.511 INFO:teuthology.orchestra.run.smithi016.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T07:18:18.559 INFO:teuthology.orchestra.run.smithi016.stderr: 94.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-05-27T07:18:18.575 INFO:teuthology.orchestra.run.smithi016.stderr: 94.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-05-27T07:18:18.602 INFO:teuthology.orchestra.run.smithi069.stderr: 91.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-05-27T07:18:18.833 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-05-27T07:18:18.894 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-05-27T07:18:22.398 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-05-27T07:18:22.780 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-05-27T07:18:25.000 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-05-27T07:18:52.070 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-05-27T07:19:11.931 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T07:19:16.246 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-05-27T07:19:22.064 INFO:teuthology.orchestra.run.smithi164.stderr: 90.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-05-27T07:19:23.867 INFO:teuthology.orchestra.run.smithi069.stderr: 90.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-05-27T07:19:32.614 INFO:teuthology.orchestra.run.smithi164.stderr: 90.8% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T07:19:40.236 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T07:19:40.240 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-05-27T07:19:40.240 INFO:teuthology.orchestra.run.smithi069.stderr:real 1m22.227s 2024-05-27T07:19:40.240 INFO:teuthology.orchestra.run.smithi069.stderr:user 3m26.649s 2024-05-27T07:19:40.240 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m15.328s 2024-05-27T07:19:57.603 INFO:teuthology.orchestra.run.smithi164.stderr: 90.6% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T07:19:57.606 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-05-27T07:19:57.606 INFO:teuthology.orchestra.run.smithi164.stderr:real 1m39.584s 2024-05-27T07:19:57.606 INFO:teuthology.orchestra.run.smithi164.stderr:user 3m51.329s 2024-05-27T07:19:57.606 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m18.740s 2024-05-27T07:23:52.534 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T07:24:23.671 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T07:25:05.953 INFO:teuthology.orchestra.run.smithi016.stderr: 90.6% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-05-27T07:25:48.968 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/3c4673a4-1bf4-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-05-27T07:26:01.842 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-27T07:26:01.846 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-05-27T07:26:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:real 7m43.843s 2024-05-27T07:26:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:user 10m28.328s 2024-05-27T07:26:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:sys 1m15.403s 2024-05-27T07:26:01.847 INFO:tasks.cephadm:Archiving logs... 2024-05-27T07:26:01.848 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi016/log 2024-05-27T07:26:01.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T07:26:53.715 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi069/log 2024-05-27T07:26:53.716 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T07:27:28.093 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi164/log 2024-05-27T07:27:28.094 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T07:28:06.767 INFO:tasks.cephadm:Removing cluster... 2024-05-27T07:28:06.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force 2024-05-27T07:28:07.302 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:28:08.922 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force 2024-05-27T07:28:09.244 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:28:10.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 --force 2024-05-27T07:28:10.979 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 3c4673a4-1bf4-11ef-bc9b-c7b262605968 2024-05-27T07:28:12.429 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T07:28:12.430 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:12.498 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:12.557 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:12.615 INFO:tasks.cephadm:Teardown complete 2024-05-27T07:28:12.616 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T07:28:12.627 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump' 2024-05-27T07:28:12.627 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T07:28:12.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T07:28:12.635 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T07:28:12.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T07:28:12.800 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T07:28:12.811 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T07:28:12.811 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:28:12.820 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:28:12.827 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:28:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:28:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:28:12.882 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:28:12.893 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:28:12.893 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-27T07:28:12.894 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 5 +886us[ +886us] +/- 56ms 2024-05-27T07:28:12.894 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -200us[ -187us] +/- 65ms 2024-05-27T07:28:12.894 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -933us[ -920us] +/- 73ms 2024-05-27T07:28:12.894 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +894us[ +887us] +/- 56ms 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -188us[ -188us] +/- 65ms 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -926us[ -926us] +/- 73ms 2024-05-27T07:28:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:28:12.975 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:28:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-05-27T07:28:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +879us[ +881us] +/- 56ms 2024-05-27T07:28:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -189us[ -188us] +/- 65ms 2024-05-27T07:28:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -927us[ -926us] +/- 73ms 2024-05-27T07:28:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:28:12.982 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T07:28:12.993 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T07:28:12.994 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T07:28:13.004 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T07:28:13.301 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T07:28:13.527 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T07:28:13.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:28:13.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:13.843 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:28:13.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:13.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:28:14.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:14.028 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T07:28:14.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T07:28:14.107 INFO:teuthology.task.internal:Duration was 3329.377817 seconds 2024-05-27T07:28:14.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T07:28:14.117 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T07:28:14.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T07:28:14.126 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T07:28:14.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T07:28:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:28:14.301 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:28:14.301 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:28:14.751 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T07:28:14.751 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-05-27T07:28:14.752 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T07:28:14.825 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-05-27T07:28:14.825 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T07:28:14.908 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-05-27T07:28:14.908 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T07:28:14.983 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T07:28:14.983 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T07:28:14.991 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T07:28:14.999 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T07:28:15.081 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T07:28:15.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T07:28:15.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T07:28:15.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T07:28:16.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T07:28:16.289 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T07:28:16.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T07:28:16.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T07:28:16.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T07:28:16.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T07:28:16.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:16.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:16.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:16.788 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-05-27T07:28:16.791 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-05-27T07:28:16.803 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-05-27T07:28:16.858 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:16.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:16.914 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:16.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:16.969 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T07:28:17.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:28:17.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T07:28:17.037 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T07:28:17.039 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi016 2024-05-27T07:28:17.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T07:28:17.173 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi069 2024-05-27T07:28:17.173 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T07:28:17.276 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727619/remote/smithi164 2024-05-27T07:28:17.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T07:28:17.380 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T07:28:17.381 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T07:28:17.389 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T07:28:17.398 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T07:28:17.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T07:28:17.466 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T07:28:17.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T07:28:17.483 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T07:28:17.483 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T07:28:17.490 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T07:28:17.498 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T07:28:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 07:28 /home/ubuntu/cephtest 2024-05-27T07:28:17.549 INFO:teuthology.orchestra.run.smithi069.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 07:28 /home/ubuntu/cephtest 2024-05-27T07:28:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 07:28 /home/ubuntu/cephtest 2024-05-27T07:28:17.558 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T07:28:17.570 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T07:28:17.584 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi164 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump' 2024-05-27T07:28:17.589 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} duration: 3329.3778173923492 failure_reason: 'Command failed on smithi164 with status 22: ''sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid 3c4673a4-1bf4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump''' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.c-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-27T07:28:17.589 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T07:28:17.667 INFO:teuthology.run:FAIL